Home
Jobs
Penta Consulting
Principal AI Test Engineer

Principal AI Test Engineer

Hybrid

Cambridge, United kingdom

Senior

Freelance

26-09-2025

Share this job:

Score my CV

Skills

CI/CD DevOps Azure DevOps Test Security Testing Azure api testing .NET CI/CD Pipelines Microsoft Azure

Job Specifications

We are seeking a meticulous and forward-thinking Principal AI Test Engineer to uphold the quality, performance, and security of our Core AI Services. You'll be responsible for validating distributed, cloud-native systems and public APIs that serve as the backbone for our enterprise AI solutions. This position requires strong technical expertise and a passion for delivering scalable, secure, and ethically sound AI capabilities.

Key Responsibilities

Conduct functional, performance, and security testing of cloud-native services hosted on Microsoft Azure.
Design, develop, and maintain automated test suites for APIs, service components, and AI workflows.
Automate the validation of AI outputs to ensure accuracy, coherence, safety, and consistency.
Work with developers and data scientists to define quality metrics and integrate observability into services.
Test systems for compliance with AI governance and regulatory frameworks, ensuring outcomes are traceable, fair, and robust.

Requirements

Minimum 10 years of experience in software QA, testing, or validation roles focused on cloud-native applications using Microsoft and .NET ecosystems.
Expertise in building and maintaining automated testing frameworks.
Hands-on experience with Azure DevOps, CI/CD pipelines, and testing in containerized environments.
Strong background in API testing, performance analysis, and security testing, including familiarity with OWASP Top 10.
Experience with LLM (Large Language Model) evaluation, scoring outputs, and applying AI validation frameworks.
Knowledge of modern AI concepts such as prompt engineering, Retrieval-Augmented Generation (RAG), model orchestration, and hallucination detection.

Please apply now if you are interested and I'll be in touch.

About the Company

At Penta Consulting, we understand that technology is only as powerful as the people behind it. As a leading provider of Technology Resource Solutions, our mission is to empower businesses with the right talent and expertise to drive their digital transformation. With over 25 years of experience, we have built a strong reputation for delivering high-quality resources across various domains, including software development, artificial intelligence, project management, architecture, cloud computing, cybersecurity, and data an... Know more