About the Company
Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe.
The Community You Will Join
AI and ML are at the heart of the Airbnb product. The Core ML team is responsible for driving Airbnb Assistance Engineering initiatives by adopting Generative AI technologies to enable an intelligent, scalable, and exceptional service experience.
The Difference You Will Make
In this Senior Staff Machine Learning Engineering role, you will set technical direction and lead execution for ML evaluation and the end-to-end data flywheel powering Airbnb Assistance Engineering.
A Typical Day
- Define evaluation strategy and success metrics for GenAI systems.
- Build and scale evaluation frameworks (golden sets, synthetic data, automated regressions, rubric-based grading, LLM-as-judge where appropriate).
- Design the data flywheel: instrumentation, feedback collection, data quality checks, labeling strategy, dataset versioning, and governance.
- Lead cross-functional quality initiatives across product, ops, and engineering.
- Develop and productionize pipelines for dataset creation, model monitoring, evaluation-at-scale, and continuous testing.
- Drive technical decisions and architecture for evaluation and data infrastructure.
Minimum Qualifications
- PhD in Computer Science, Mathematics, Statistics, or related technical field (or equivalent practical experience).
- 10+ years building, testing, and shipping ML/AI systems end-to-end; including 2+ years of experience with GenAI/LLM systems in production.
- 5+ years leading large, ambiguous technical initiatives as a senior IC.
- Deep expertise in evaluation methodology (offline/online alignment, metric design, human-in-the-loop evaluation, A/B testing, power analysis, regression testing).
- Hands-on experience with GenAI systems, including orchestration, retrieval, tool calling, memory, etc.
- Experience building data pipelines and quality systems.
- Solid ML fundamentals and best practices.
Preferred Qualifications
- Experience applying ML/AI to customer support workflows.
- Experience building robust evaluation platforms for agent behavior validation, safety/guardrails, and continuous improvement.
- Proven ability to take evaluation and data flywheel work from incubation to production.
Your Location
This position is US - Remote Eligible. The role may include occasional work at an Airbnb office or attendance at offsites, as agreed to with your manager. While the position is Remote Eligible, you must live in a state where Airbnb, Inc. has a registered entity.