7+ years of experience building and shipping production software, ideally including LLM-powered agents that take real actions in a product.
Experience with complex, tool-using systems involving multiple tools, planning or orchestration, and sub-agents—not just simple, single-turn assistants. Be able to explain something you shipped and how you knew it was working.
Comfortable with Ruby on Rails and/or Python, or ready to pick them up quickly. Ruby on Rails is the production system and the foundation that Circle’s AI Agents are built on. Proficiency in Python is a strong plus, especially for data and evaluation work.
Experience building evaluation or observability infrastructure for ML/AI systems, such as eval pipelines, scorers, dashboards, or CI/CD for evals. Experience with evaluation frameworks such as Braintrust, LangSmith, or similar.
Experience designing datasets, annotation workflows, or labeling pipelines for ML/AI evaluation. Ability to turn raw examples into a trustworthy dataset through human annotation, AI-generated data, or simulation.
Learn fast, experiment aggressively, and thrive in a highly dynamic environment. Comfortable running many experiments and letting results settle the argument, including experiments that do not work, which will be many at the beginning.
Comfortable in a fast-paced environment with ambiguity; able to learn and pick up new technologies when projects require it.
Proficient in English (spoken, written, and reading) at CEFR Level C2 / ILR Level 5.