Senior Machine Learning Engineer: Search Quality

About the role

This role focuses on advancing the search capabilities of Constructor's AI-first e-commerce platform. The core work involves using LLMs to evaluate search relevance and user intent, then fine-tuning these models to improve product discovery. A key aspect is ensuring these quality enhancements are integrated into production systems with strict latency requirements, impacting how shoppers find products.

The position also requires developing and maintaining automated pipelines for monitoring search quality. This includes detecting performance degradations and generating recommendations for improvements. Success in this role means contributing to a system where shoppers consistently find relevant products, directly impacting revenue and conversion rates for global e-commerce brands.

You will be part of a fully remote team, collaborating with ranking, product, and data engineering teams. The work environment emphasizes growth and impact, supported by a culture that encourages taking time off and provides resources for professional development. This role is suited for someone who thrives in a collaborative, technically challenging setting focused on practical applications of machine learning.

The base salary for this position ranges from $115,000 to $195,500 USD annually, with stock options offered in addition.

Skills that matter here

• Python: This role requires strong proficiency in Python for developing and deploying machine learning systems.
• PyTorch: Hands-on experience with PyTorch is essential for fine-tuning and scaling LLM deployments.
• SQL: Fluency in SQL is necessary for data manipulation and analysis within the ML pipelines.
• Spark: Experience with Spark is needed for data orchestration, particularly in processing large datasets.
• Airflow: This role uses Airflow for managing and automating data workflows and ML pipelines.

Who this role suits

• You have a background shipping production machine learning systems, specifically in search, information retrieval, or recommendation.
• You are skilled in fine-tuning and evaluating large language models and can scale their deployments.
• You are adept at designing and executing A/B tests to measure the impact of model changes.
• You enjoy collaborating with diverse teams, including product and data engineering, to achieve project goals.

From the employer

• LLM-based evaluation: fine-tune models to assess relevance, understand query intent, and align judgments with user behavior.
• Real-time quality in production: reranking, filtering, and signal computation with strict latency requirements.
• Automated quality monitoring: pipelines to detect degradations and generate actionable recommendations.

• 4+ years shipping production ML systems
• Experience with search, information retrieval, or recommendation systems
• Hands-on experience with fine-tuning, evaluation frameworks, and scaling LLM deployments
• Strong Python and PyTorch; fluency in SQL and data orchestration tools (Spark, Airflow)
• Experience designing and running A/B tests to validate model impact
• Excellent English communication skills
• Experience collaborating in cross-functional teams (ranking, product, data engineering)

• Work with smart and empathetic people who will help you grow and make a meaningful impact.
• Regular team offsite events to connect and collaborate.
• Fully remote team - choose where you live.
• Unlimited vacation time - we strongly encourage all of our employees take at least 3 weeks per year.
• Work from home stipend for home office setup.
• Apple laptops provided for new employees.
• Training and development budget refreshed each year.
• Maternity & Paternity leave for qualified employees.
• Base salary: $80k–$120K USD, depending on knowledge, skills, experience, and interview results.
• Stock options offered in addition to the base salary.