About the company
At Anyscale, we're on a mission to democratize distributed computing and make it accessible to software developers of all skill levels. We're commercializing Ray, a popular open-source project that's creating an ecosystem of libraries for scalable machine learning. Companies like OpenAI, Uber, Spotify, Instacart, Cruise, and many more, have Ray in their tech stacks to accelerate the progress of AI applications out into the real world. Proud to be backed by Andreessen Horowitz, NEA, and Addition with $250+ million raised to date.
Responsibilities
- Design, build, and improve the core systems that power Ray Data, with a focus on performance, scalability, and reliability
- Design and optimize distributed execution across different stages of data pipelines in heterogeneous environments
- Build data loading and processing solutions for production training and inference workloads
- Solve challenging problems in distributed execution, scheduling, resource management, data partitioning, fault tolerance, and performance optimization
- Make system-level architectural decisions and reason through tradeoffs in areas such as resource allocation, execution models, batch vs. streaming workloads, and consistency and availability
- Work with customers and new-age AI-native companies to understand and solve challenges in scaling their AI workloads
Requirements
- 6+ years of experience building production-grade software, infrastructure, or developer-facing systems, with strong Python engineering experience
- 6+ years of experience personally owning core architectural decisions within a distributed data or compute engine
- Deep experience with distributed systems internals, such as scheduling, fault tolerance, data partitioning, distributed execution, performance optimization, or database and query engine internals
- A track record of reasoning through system-level tradeoffs and defending architectural decisions
- Passion for solving the unsolved problems in large-scale AI infrastructure
Conditions
- Competitive salary and equity, plus health/dental/vision coverage (many plans up to 99% employer-covered)
- Flexible time off, paid parental leave, and mental health support
- Equal Opportunity Employer