Generative AI Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Generative AI Engineer based in United States.

• Design, develop, test, deploy, and maintain generative AI solutions that ground large language model responses in authoritative healthcare program documentation.
• Build and maintain document ingestion pipelines covering parsing, chunking, metadata tagging, embedding generation, and vector or semantic search indexing.
• Translate user needs and stakeholder requirements into effective prompts, context strategies, system requirements, and evaluation criteria in collaboration with cross-functional teams and subject matter experts.
• Implement AI responses that include source citations and direct links to underlying documentation, ensuring that users can trace answers back to authoritative materials.
• Design and implement responsible AI guardrails, including content filtering, scope restrictions, PII and PHI protections, and appropriate refusal handling for out-of-scope questions.
• Develop evaluation frameworks and test datasets to measure retrieval quality, answer accuracy, groundedness, consistency, and overall system performance.
• Continuously monitor, evaluate, and tune AI solutions to improve reliability while balancing accuracy, latency, cost, security, and specific use-case requirements.
• Design AI applications capable of supporting multiple foundation models and providers and evaluate model performance against business and technical requirements.
• Deploy and operate solutions using approved AWS services and within applicable federal security controls, contributing to documentation required for security authorization and AI governance reviews.
• Implement logging, monitoring, and audit trails that support transparency, operational oversight, cost management, and responsible AI governance.
• Write clean, reusable, well-documented code and contribute to code reviews, automated testing, deployment processes, and ongoing system maintenance.
• Collaborate with technical and non-technical stakeholders to communicate AI capabilities, limitations, risks, and performance clearly.
• Stay current with emerging generative AI technologies, foundation models, development frameworks, security practices, and federal AI guidance.

• You have a master’s degree in computer science, data science, engineering, or another relevant technical field.
• You have at least 5 years of experience developing software and cloud-based solutions, including recent hands-on experience designing and deploying LLM or generative AI applications.
• You have strong proficiency in Python and experience working with modern LLM APIs, SDKs, orchestration frameworks, and AI/ML libraries such as Hugging Face, LangChain, LlamaIndex, or equivalent technologies.
• You have hands-on experience designing and deploying generative AI systems involving embeddings, vector databases, and hybrid or semantic search.
• You have experience with AWS AI and data services such as Amazon Bedrock, Amazon OpenSearch, Amazon Kendra, SageMaker, Lambda, or S3.
• You have practical experience with prompt engineering, context engineering, LLM evaluation, and techniques for reducing hallucinations and ensuring responses remain grounded in source material.
• You understand how to evaluate and select foundation models based on factors including accuracy, latency, cost, security, and specific application requirements.
• You have experience working within Agile software development environments and understand modern software engineering and delivery practices.
• You have experience building and maintaining CI/CD pipelines and deploying infrastructure using tools such as GitHub Actions, CloudFormation, Terraform, and/or Jenkins.
• You have strong problem-solving, communication, and collaboration skills, including the ability to explain complex AI capabilities and limitations to non-technical stakeholders.
• You can work effectively both independently and as part of a multidisciplinary team, with strong attention to detail and a professional approach to documentation and presentation.
• Experience supporting federal health programs, including program policy, regulatory guidance, or customer and stakeholder support operations, is highly desirable.
• Experience deploying AI solutions in federal environments, including FedRAMP-authorized services, ATO or security authorization processes, and NIST 800-53 controls, is preferred.
• Familiarity with responsible AI and federal AI governance frameworks, including the NIST AI Risk Management Framework and OMB AI guidance, is advantageous.
• Knowledge of healthcare security and compliance requirements such as HIPAA and HITECH is preferred.
• Experience building conversational or search interfaces, including familiarity with Section 508 accessibility requirements, is a plus.
• Relevant AWS certification, such as AWS Certified Machine Learning Engineer – Associate, AWS