AI Agent Evaluation Engineer

Job Title:

AI Agent Evaluation Engineer
Experience Level:

5-7 Years (6+ years in Software QA required)
Work Split:

70% Automation / 30% Manual Testing
Key Focus:

Responsible AI, Safety Evals, and Google ADK

AI/LLM Testing Experience:

Minimum 2 years focused specifically on testing/evaluating AI systems, conversational agents, or LLMs.
Safety & Red Teaming:

Direct experience in Safety Evals is mandatory. This includes red teaming, adversarial testing, jailbreaking, and measuring toxicity/bias.
Google ADK Knowledge:

Must have direct experience or a strong conceptual understanding of the

Google Agent Development Kit (ADK)

and Vertex AI.

Technical Stack:

o Strong proficiency in

Python

for scripting and automation.
o Experience with

PyTest .
o

Prompt Injection

experience.

Tooling Familiarity:

Experienc...