AI Scientist

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior AI Scientist based in Portugal.

• Analyze AI conversations, interaction data, and user feedback to identify opportunities to improve response quality, accuracy, relevance, and overall user experience.
• Design, implement, test, and optimize prompt engineering strategies and conversational behaviors.
• Optimize RAG pipelines, retrieval mechanisms, vector-based search, and overall retrieval quality.
• Evaluate different LLMs and recommend the most effective models for specific product use cases, including exploring the development or training of customer-specific models where appropriate.
• Define, track, and continuously improve AI quality metrics such as accuracy, relevance, latency, and user satisfaction.
• Design and execute structured experiments to assess AI improvements and determine their impact.
• Develop and maintain evaluation frameworks, benchmarks, testing methodologies, and supporting documentation.
• Collaborate closely with AI Engineers to validate and deploy successful improvements into production environments.
• Monitor the performance of production AI systems and identify opportunities for optimization at scale.
• Stay informed about advances in generative AI, conversational AI, speech and voice technologies, and recommend practical applications that can improve existing products.
• Help ensure AI solutions are developed and evaluated according to appropriate governance, safety, and responsible AI principles.

• 5+ years of professional experience in AI, machine learning, NLP, or a related technical field.
• Strong practical understanding of LLMs, generative AI, and conversational AI systems.
• Hands-on experience with prompt engineering, prompt evaluation, and LLM optimization.
• Proven experience evaluating conversational AI using both qualitative and quantitative methodologies.
• Strong Python and SQL skills, with the ability to analyze data and build practical evaluation or experimentation workflows.
• Experience with AI evaluation frameworks, benchmarking, and systematic model or system assessment.
• Knowledge of RAG architectures, vector databases, embedding models, and retrieval optimization.
• Experience working with production AI systems at scale.
• Experience with speech and voice AI technologies is highly valuable.
• Understanding of AI governance, safety, and responsible AI practices.
• Strong analytical and problem-solving skills, combined with a structured and experimental approach to complex challenges.
• Excellent communication and stakeholder management skills, with the ability to translate technical findings into clear recommendations.
• Proactive mindset, strong ownership, and accountability for measurable outcomes.

• Competitive salary with individual performance-based bonuses paid quarterly.
• 28 days of paid annual leave.
• Remote-first working environment with flexibility across your local time zone.
• Core working hours from 10:00 AM to 3:00 PM local time, with flexibility outside those hours.
• Referral bonuses and additional flash bonus opportunities.
• High-quality, top-of-the-line work equipment.
• Annual company retreats designed to strengthen collaboration and internal networking.
• Opportunity to work on impactful AI products and contribute to the evolution of generative and conversational AI.
• International, remote-friendly environment with significant autonomy and flexibility.