- Company Name
- Imbibe Consultancy Services Pvt Ltd
- Job Title
- AI/ML Test Engineer
- Job Description
-
Job title: AI/ML Test Engineer
Role Summary: Senior engineer responsible for validating AI/ML models and data‑driven applications (NLP, recommendation engines, fraud detection, advanced analytics). Ensures accuracy, bias mitigation, explainability, robustness, and overall performance across end‑to‑end ML pipelines. Integrates testing within CI/CD pipelines and monitors model drift.
Expectations: Deliver rigorous, reproducible test outcomes that satisfy business rules and compliance. Collaborate with data scientists, ML engineers, and product owners to define test strategies, resolve quality issues, and continuously improve the testing framework. Maintain documentation, metrics, and audit trails for AI decisions.
Key Responsibilities:
- Test and validate AI/ML models for accuracy, bias, explainability, robustness, and performance.
- Perform data validation on training, testing, and inference datasets to meet quality and business requirements.
- Validate end‑to‑end ML pipelines: data ingestion, feature engineering, training, deployment, and inference.
- Design and execute AI‑specific test strategies, including boundary, adversarial, and drift testing.
- Define and implement AI/ML testing frameworks and automation tools.
- Integrate AI testing into CI/CD pipelines for continuous validation.
- Identify and document model misuse scenarios (e.g., manipulation of recommendations, fraud detection bypass).
- Collaborate with stakeholders to translate business and procurement rules into test cases.
Required Skills:
- 5+ years of experience testing AI/ML models and data‑driven applications.
- Expertise in NLP, recommendation engines, fraud detection, and analytics models.
- Strong knowledge of bias detection, explainability, and robustness techniques.
- Proficiency with Python, PyTest, TensorFlow Model Analysis, MLflow, and data‑validation libraries.
- Familiarity with CI/CD tooling (Git, Jenkins, GitLab, etc.) and automation of AI tests.
- Ability to design test strategies for boundary and adversarial scenarios, and monitor model drift.
- Excellent analytical, problem‑solving, and communication skills.
Required Education & Certifications:
- Bachelor’s or Master’s degree in Engineering, Computer Science, Information Technology, or Data Science (or equivalent).
- ISTQB AI Testing certification, TensorFlow/Cloud AI certification, or comparable AI testing credential (preferred).