Home
Jobs
The Brixton Group
Generative AI Quality Engineer

Generative AI Quality Engineer

Hybrid

California, United states

Freelance

03-03-2026

Share this job:

Score my CV

Skills

Java ServiceNow CI/CD Monitoring Jenkins Test Test Automation Performance Testing Regression Autonomy CI/CD Pipelines

Job Specifications

Duration: 3+ months
Location: San Francisco, CA

We’re seeking a Generative AI Quality Engineer to ensure the reliability, accuracy, and ethical integrity of AI-driven solutions. This role focuses on automated test strategy, validation of AI model outputs, and performance monitoring across chatbot use cases.

Requirements:
Strong test automation experience in enterprise environments
Hands-on development in Java + Playwright
Experience building and maintaining automation frameworks from scratch
AI-assisted coding experience (e.g., Windsurf or similar tools)
CI/CD pipeline integration & maintenance (Jenkins)
Proven success in a high-autonomy, self-directed environment
Experience validating AI/chatbot output quality, accuracy, and edge cases
Responsibilities:
Design and execute automated test strategies for generative AI models
Build scalable automation suites using Java + Playwright
Test and validate AI chatbot functionality, response accuracy, bias, and reliability
Develop regression, integration, and performance testing frameworks
Integrate automation into CI/CD pipelines (Jenkins)
Monitor AI output quality across diverse use cases
Collaborate cross-functionally with AI/ML, Product, and Engineering teams
Nice to Have:
Experience testing AI chatbots / conversational AI platforms
ServiceNow platform experience
Exposure to AI model evaluation frameworks

26-00248

About the Company

We are a woman-owned values-based professional technology services firm with over 150 years of combined experience in the technology industry. We’re deeply engaged in the successful pairing of the right people to the right projects, and we attribute our national presence to the referrals we’ve received through current and past clients and candidates. We believe that when people find the ideal setting to express their talents, the possibilities are infinite. This is why we exist. We are inspired and guided by a greater purpos... Know more