SeniorRemote
Member of Technical Staff, ML Platform
О роли
Описание вакансии
About the company
- Runway builds AI to simulate the world by merging art and science
- World models at the frontier of AI progress
- Remote hiring open
Responsibilities
- Own the evaluation platform end to end: tooling and systems to generate, annotate, review and adapt at frontier scale
- Define CLIs, APIs, GUIs and storage layers for a seamless, fast, collaborative process
- Work with research teams on video, image, audio, agents and robotics to understand what to measure, then generalize into the platform
- Set standards for model evaluation: reproducibility, metric definitions, reporting formats, trustworthiness
- Support broad adoption of the platform across training and production model serving
- Contribute to ML Platform tools and systems
Requirements
- 5+ years building ML infrastructure or data platforms in production, including evaluation, experimentation or benchmarking systems
- Strong Python and PyTorch, hands-on experience running large batch GPU workloads on Kubernetes
- Experience designing data pipelines and storage for large volumes of media or model outputs, with versioning and reproducibility
- Familiarity with experimental statistics: paired comparisons, confidence intervals, multiple-comparison pitfalls, inter-rater agreement
- Comfort building internal tools end to end, from CLI to browser
- Ability to lead a broad technical area: gather requirements, set direction, make tradeoffs, drive a roadmap
- Familiarity with the full model development lifecycle: data, training, evaluation, serving
- Strong systems thinking and pragmatic approach to production reliability
Nice to have
- Track record of building evaluation suites for generative models
- Hands-on work with LLM- or VLM-as-judge pipelines
- Experience with online experimentation platforms and connecting offline metrics to product outcomes
- Prior work evaluating agents or robotics policies
Стек и навыки
С чем работаем