About the company
It all started when engineer Fred Luddy wrote code that automated a tedious task for his coworker, Phyllis. She cried tears of joy. That moment inspired Fred to build a company that could do that for everyone—freeing people from busywork so they could focus on meaningful work. Today, ServiceNow is the AI control tower for business reinvention. Our ServiceNow AI platform brings together any AI, any data, and any workflow— helping 85% of the Fortune 500® work smarter, faster, and better. We're building an AI-native culture where technology and talent are unstoppable together. And we're just getting started.
Join us to put AI to work for people.
Responsibilities
- Design and architect data infrastructure: Design and oversee deployment of the data architecture and pipelines that capture, manage, and store structured and unstructured data from internal and external sources. Establish the processes and data flows across cloud services, local databases, and other applicable storage forms, and own the contracts between data producers and consumers.
- Build and automate data transformation: Develop technical tools using machine learning and data-engineering techniques to cleanse, organize, and transform data. Implement automated processes that maintain the integrity of data structures and hold quality standards on an ongoing basis rather than at a point in time.
- Define agentic evaluation metrics and ground truth: Partner with product and AI teams to define evaluation metrics for agentic workflows, including task and mission completeness, instruction adherence, tool use, and end-to-end workflow success. Establish ground truth labeling standards, annotation guidelines, and validation criteria, and design evaluation datasets that reflect real-world agent execution rather than idealized paths.
- Build agentic evaluation pipelines: Design and implement automated evaluation infrastructure that measures AI agent performance using LLMs and agent execution logs. Create the dashboards, reporting, versioning, and reproducibility that make evaluation datasets and results trustworthy over time and comparable across releases.
- Establish standards and continuous improvement: Create design standards and quality assurance processes for data systems. Define quality gates and validation frameworks, analyze workflow performance, and recommend optimizations that keep the platform aligned with evolving CRM AI requirements.
- Lead cross-functional collaboration: Collaborate with product and AI teams to define evaluation metrics, set labeling standards, and build datasets that reflect real-world agent behavior.