Quality & Evaluation Engineer - GenAI Innovation Team

About The Position

Company Overview:

About the Job

We are looking for an AI Quality & Evaluation Engineer to own the quality planning and execution of an AI-powered chat application operating over complex law enforcement and mobile device data.

This is a highly hands-on role focused on execution rather than high-level QA strategy. You will design, build, and run automated and semi-automated tests for LLM-driven workflows, create evaluation datasets, and continuously stress the system with realistic and extreme investigative scenarios.

What you will be doing

• Design, plan, and execute quality tests for an AI chat application built on LLMs and investigative data.

• Build and maintain automation frameworks for prompt regression testing, multi-turn conversations, and model upgrades.

• Create and curate evaluation datasets used for regression testing, benchmarking, and model comparison.

• Design c...