- Company Name
- 49 Solutions
- Job Title
- AI Architect / Tester
- Job Description
-
Job title: AI Architect / Tester
Role Summary: Design, develop, and execute testing strategies for AI applications (agents, RAG, LLMs) to ensure functionality, performance, reliability, and bias mitigation across multiple federal projects.
Expectations: Deliver robust test plans, maintain evolving test cases, validate ML models and frameworks, implement automation, support CI/CD, and comply with federal security and best‑practice standards.
Key Responsibilities:
- Develop test strategies, plans, and cases for AI solutions.
- Conduct functional, performance, integration, and bias testing.
- Create and maintain automated tests using Python, Java, or R.
- Evaluate and validate models/algorithms in TensorFlow, Keras, PyTorch.
- Collaborate with developers, data scientists, and cross‑functional teams to resolve issues.
- Support DevOps and CI/CD pipelines for AI deployment.
- Ensure security, quality, and compliance with federal guidelines.
Required Skills:
- 5+ years software testing/test automation, AI focus.
- Strong knowledge of AI agents, RAG, LLMs, ML models.
- Proficiency in Python, Java, or R for automation.
- Experience with TensorFlow, Keras, PyTorch, and AI testing tools.
- Familiarity with DevOps, CI/CD, and related tooling.
- Analytical, problem‑solving, and communication abilities.
Required Education & Certifications:
- Bachelor’s degree in Computer Science, Engineering, or related field.
- Valid Canadian Government Security Clearance.