Senior AI Evals Engineer

Описание

Senior AI evals test engineering for LLM agents built on Google ADK and similar agentic frameworks; 70% automation, 30% manual.

Responsibilities

• Design and build evaluation frameworks for LLM-based agents across correctness, latency, cost, safety and task completion
• Build synthetic test environments, golden datasets and CI-integrated regression and drift detection for AI agents

Requirements

• 8+ years in software test engineering, with 5+ years in QA automation or developer tooling and 2+ years on production ML/AI systems
• Hands-on LLM evaluation experience with tools such as DeepEval, LLM-as-a-Judge, Promptfoo, RAGAS or LangSmith
• Strong Python scripting, pytest and CI/CD experience
• Familiarity with GCP or another cloud for AI/ML

We offer

• ₹24 - 70 LPA (estimated)