- Company Name
- Cleo
- Job Title
- Lead Machine Learning Engineer, Chat
- Job Description
-
**Job Title:** Lead Machine Learning Engineer, Chat
**Role Summary:**
Lead the design, deployment, and continuous improvement of machine learning solutions for Cleo’s chat evaluation features. Drive technical strategy, mentor a small team, and collaborate closely with product to enhance chatbot observability and AI development cycles.
**Expectations:**
- 5+ years in data science, ML engineering, or related roles.
- Proven track record of deploying multiple production‑grade ML models.
- Experience integrating and evaluating large language models (LLMs).
- Strong ability to translate analytical insights into actionable product stories and communicate them to non‑technical stakeholders.
- Ability to create short‑term roadmaps, manage project plans, and report progress to leadership.
- Comfortable breaking tasks into incremental, testable components.
**Key Responsibilities:**
- Design and implement ML models that improve chatbot quality, user engagement, and payment success rates.
- Fine‑tune LLMs using user interaction data and develop ranking systems for conversational context.
- Build and maintain end‑to‑end pipelines for data ingestion, feature engineering, model training, and deployment.
- Collaborate with product, engineering, and data teams to translate business needs into measurable ML objectives.
- Mentor junior engineers, review code, and establish best practices for model quality and scalability.
- Monitor deployed models’ performance, detect drift, and iterate on models and features.
- Investigate and prototype new techniques such as LLM‑as‑a‑judge or advanced annotation pipelines.
- Provide clear documentation, demos, and performance reports to stakeholders.
**Required Skills:**
- Python, SQL, and strong knowledge of statistical fundamentals.
- Production‑ready ML experience: model training, deployment pipelines, monitoring, and scaling.
- Familiarity with Docker and container orchestration for model hosting.
- Experience with large language models, including fine‑tuning and evaluation.
- Ability to communicate complex findings to non‑technical audiences.
- Project leadership: roadmap development, incremental planning, and cross‑functional coordination.
**Nice to Have:**
- Experience in LLM‑as‑a‑judge setups or annotation pipelines.
- Familiarity with distributed training tools and cloud ML platforms.
**Required Education & Certifications:**
- Bachelor’s degree in Computer Science, Statistics, Engineering, or a related field (Master’s or higher preferred).
- Relevant certifications (e.g., TensorFlow, PyTorch, or cloud ML services) are advantageous but not mandatory.