Senior AI Test Engineer / AI Quality Engineer

Описание

This role is for an experienced AI Test Engineer with strong expertise in AI/ML application testing, Generative AI validation, LLM evaluation, and test automation. The ideal candidate will have hands-on experience testing AI-powered applications, including Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) pipelines, conversational AI, and agentic AI workflows. The role requires strong Python-based automation skills, experience with Selenium or Playwright, API testing, and the ability to build scalable AI quality validation frameworks.

Responsibilities

• Testing AI/ML applications, Generative AI validation, LLM evaluation, and test automation
• Testing AI-powered applications, including Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) pipelines, conversational AI, and agentic AI workflows
• Building scalable AI quality validation frameworks

Requirements

• Strong hands-on experience with AI/GenAI application testing and LLM evaluation
• Experience with hallucination detection, response accuracy, relevance, faithfulness, and prompt validation
• Hands-on experience testing RAG pipelines, including retrieval accuracy and groundedness
• Proficiency in Python for AI test automation and validation
• Experience with DeepEval, RAGAS, Promptfoo, or similar LLM evaluation frameworks
• Strong experience with Selenium or Playwright for UI automation
• Experience with REST API testing using Postman, REST Assured, or Python-based frameworks
• Experience validating AI applications integrated with enterprise systems
• Knowledge of CI/CD pipelines using Jenkins, GitHub Actions, or Azure DevOps
• Experience with Agile/Scrum methodologies and defect management
• 8+ years in QA Automation, including hands-on AI/GenAI testing experience

Nice to have

• Experience in the Insurance or Financial Services domain is highly preferred

We offer

• Long-Term Contract (6-12+ months)
• Salary: $55 - $70 per hour