ServiceNow is the AI control tower for business reinvention, helping 85% of the Fortune 500 work smarter, faster, and better.
Build Agent is ServiceNow's AI coding assistant, purpose-built for the platform's metadata-driven substrate.
Responsibilities
Own eval strategy and roadmap: prompt set design, scoring methodology, failure-mode taxonomy, and telemetry instrumentation.
Coordinate across the multiple teams contributing to Build Agent to arbitrate quality standards and drive eval findings to committed, owned fixes.
Lead model benchmarking efforts, making data-backed recommendations on model support decisions.
Solve ambiguous, cross-cutting problems where eval signal, model capability, and platform architecture intersect.
Requirements
Direct experience building or leading evaluation systems for LLM-based products — scoring methodology, benchmark design, and failure analysis.
Understanding of coding agent architecture: tool use, agentic loops, context management, and trade-offs between generic and domain-specific scaffolding.
6+ years of experience with technologies relevant to ServiceNow, including advanced coding skills and fluency in one or more of Java, C++, Ruby, Shell, or JavaScript.
Experience critically evaluating foundation models.
Ability to execute against ambiguous priorities, weighing context, risk, and desired outcomes.
Conditions
Full-time, Regular employee type.
Work Persona: Flexible.
Base pay $171,900 - $300,800 plus equity, variable/incentive compensation and benefits.