Job Description
Job Title: AI Agent Evaluation Engineer
Experience Level: 5-7 Years (6+ years in Software QA required)
Work Split: 70% Automation / 30% Manual Testing
Key Focus: Responsible AI, Safety Evals, and Google ADK
AI/LLM Testing Experience: Minimum 2 years focused specifically on testing/evaluating AI systems, conversational agents, or LLMs.
Safety & Red Teaming: Direct experience in Safety Evals is mandatory. This includes red teaming, adversarial testing, jailbreaking, and measuring toxicity/bias.
Google ADK Knowledge: Must have direct experience or a strong conceptual understanding of the Google Agent Development Kit (ADK) and Vertex AI.
Technical Stack:
o Strong proficiency in Python for scripting and automation.
o Experience with PyTest.
o Prompt Injection experience.
Tooling Familiarity: Experience with libraries such as Langsmith, DeepEval, Ragas, Giskard, or Hugging Face.
Responsibilities:
- Develop synthetic testing environments and simulation s...
Experience Level: 5-7 Years (6+ years in Software QA required)
Work Split: 70% Automation / 30% Manual Testing
Key Focus: Responsible AI, Safety Evals, and Google ADK
AI/LLM Testing Experience: Minimum 2 years focused specifically on testing/evaluating AI systems, conversational agents, or LLMs.
Safety & Red Teaming: Direct experience in Safety Evals is mandatory. This includes red teaming, adversarial testing, jailbreaking, and measuring toxicity/bias.
Google ADK Knowledge: Must have direct experience or a strong conceptual understanding of the Google Agent Development Kit (ADK) and Vertex AI.
Technical Stack:
o Strong proficiency in Python for scripting and automation.
o Experience with PyTest.
o Prompt Injection experience.
Tooling Familiarity: Experience with libraries such as Langsmith, DeepEval, Ragas, Giskard, or Hugging Face.
Responsibilities:
- Develop synthetic testing environments and simulation s...
Apply for this Position
Ready to join Brillio? Click the button below to submit your application.
Submit Application