ML Researcher

Toloka creates data that powers leading GenAI models and innovations. It works with frontier labs, big tech, AI startups, enterprises, and non-profit research organizations worldwide. Its ML team builds machine-learning products for the platform, including post-training, fine-tuning and RL, evaluation and benchmarking, and LLM-based quality assurance.

• Define the multi-quarter strategy and architecture for Toloka’s post-training stack, including fine-tuning, RL, and evaluation capabilities.
• Lead the development of advanced post-training and RL pipelines, including GRPO, PPO/DPO, reward modeling, Process Reward Models, and RLAIF for LLM alignment and agent behavior.
• Design and calibrate automated evaluation ecosystems, including human-aligned LLM-as-a-judge frameworks, dynamic benchmarking suites, and regression controls.
• Lead research on model distillation, quantization, context compression such as gisting, and speculative decoding to optimize latency-cost trade-offs.
• Architect autonomous guiding agents and tool-use workflows for multi-step reasoning, self-correction, and evaluation-driven feedback loops.
• Provide technical leadership through architectural design reviews, mentorship of Senior ML Researchers, and reproducible ML R&D practices.
• Partner with Product and Engineering directors on scalable platform architecture; author research write-ups and blog posts, and give external tech talks.

• 6+ years in ML engineering or applied research, including 3+ years leading LLM post-training, fine-tuning, or alignment initiatives at scale.
• Deep theoretical and practical mastery of LLM alignment: SFT, LoRA/PEFT, RLHF/RLAIF (GRPO, DPO, PPO), reward modeling, and reasoning-oriented post-training.
• Experience building evaluation pipelines from scratch, addressing LLM-as-a-judge biases, and calibrating automated evaluations against human gold standards.
• Strong proficiency in Python, PyTorch, distributed training frameworks (DeepSpeed, Megatron, FSDP), and high-performance inference engines (vLLM, TensorRT-LLM).
• Ability to turn ambiguous technical challenges into robust platform features while balancing research, production reliability, and cost constraints.
• Experience as a technical leader, mentoring senior researchers or engineers, and influencing cross-functional roadmaps.
• Fluent spoken and written English (C1), with the ability to communicate complex technical concepts to internal teams and external clients or the community.

• International, dynamic environment focused on innovation in AI and technology.
• Competitive compensation package including base salary, bonus, and ESOP.
• Paid PTO and benefits vary by location.
• Full remote or hybrid model if based in the Netherlands or Serbia.
• IT setup and home office allowances.