About the company
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Quality Assurance Engineer based in India.
Responsibilities
- Build and maintain automated end-to-end, component, API, and contract test suites using Python and modern browser automation frameworks, with parallel execution, trace-based debugging, and reliable test architecture
- Establish quality gates within CI/CD pipelines, including parallelized test execution, risk-based test selection, and automated promotion criteria
- Provision ephemeral, containerized test environments using infrastructure-as-code and support synthetic or masked test-data generation
- Monitor test reliability, track flaky tests, improve suite stability, and maintain defined performance and runtime expectations
- Validate graph databases and analytics, including schema integrity, relationships, queries, data ingestion, deduplication, and graph-based risk or fraud analysis
- Develop evaluation frameworks for AI/ML systems, including accuracy thresholds, model drift, fairness, explainability, and regression testing
- Design automated validation for agentic workflows, covering tool calling, orchestration, context management, failure recovery, guardrails, and non-deterministic outputs
- Implement testing strategies such as golden datasets, semantic similarity, rubric-based evaluation, and LLM-assisted assessment where appropriate
- Collaborate with engineers and data science teams on shift-left quality, feature-level test coverage, root-cause analysis, and feature-pipeline validation
- Improve development, testing, and deployment processes, support application onboarding to standardized operating models, and document and escalate issues effectively
Requirements
- Bachelor’s degree in Computer Science or a related field
- Proven hands-on experience designing, developing, and executing automated tests for web applications and services using Python and a modern browser automation framework such as Playwright, Selenium WebDriver, or an equivalent
- Strong understanding of maintainable test architecture, including Page Object Model or comparable patterns
- Extensive Python scripting experience, with additional scripting knowledge in Unix shell, Ruby, or similar technologies
- Experience testing web services and REST APIs using tools such as Postman or Swagger
- Working knowledge of graph databases such as Neo4j, Amazon Neptune, TigerGraph, or JanusGraph, including writing and validating graph queries
- Experience with SQL and continuous integration platforms such as Jenkins, GitHub Actions, or GitLab CI
- Understanding of testing AI/ML-backed features and the differences between deterministic and probabilistic validation approaches
- Ability to identify process improvements and communicate technical findings clearly across engineering and cross-functional teams
- Experience with agentic frameworks such as LangChain, LangGraph, MCP, or comparable technologies is advantageous
- Familiarity with LLM evaluation practices, golden datasets, rubric-based scoring, hallucination and jailbreak testing, and red-teaming is beneficial
- Exposure to graph analytics libraries, observability-driven testing, Docker, Kubernetes, Terraform, Ansible, microservices, cloud APIs, and issue-tracking platforms such as JIRA is a plus
Conditions
- Comprehensive health and wellness plans
- Paid time off and company holidays
- Flexible, remote-friendly working opportunities
- Maternity and paternity leave
- Opportunity to work on advanced technologies spanning AI/ML, graph analytics, agentic systems, cloud infrastructure, and automation
- Collaborative environment focused on continuous improvement and modern engineering practices
- Inclusive workplace committed to equal employment opportunities and diverse teams