- Company Name
- Centraprise
- Job Title
- Azure DevOps Consultant (Databricks, Pyspark)
- Job Description
-
**Job Title:**
Azure DevOps Consultant (Databricks, Pyspark)
**Role Summary**
Provide end‑to‑end Azure cloud infrastructure and data engineering support for a 12+ month contract. Design, deploy, and maintain Azure resources using Terraform and ARM templates, build and manage data pipelines with Databricks, PySpark, ADF, and SQL, and integrate OpenAI models. Implement CI/CD workflows in Jenkins, version control in Git, and support related DevOps practices across Azure services.
**Expectations**
- 12+ month contract with flexible delivery schedule.
- Deliver fully functional Azure environments and data pipelines that meet operational and security standards.
- Maintain high quality documentation and stay current on Azure and Databricks best practices.
**Key Responsibilities**
- Provision and configure Azure resources (VMs, networking, storage, function apps, web apps) via Terraform and ARM templates.
- Administer and develop Databricks notebooks and clusters, including workload management and data versioning.
- Design and implement Azure Data Factory pipelines and SQL-based data transformations.
- Deploy and test OpenAI models in Azure AI Studio, ensuring proper integration with data services.
- Build and maintain Jenkins CI/CD pipelines for automated build, test, and deployment processes.
- Manage Git repositories, enforce branching strategies, and perform code reviews.
- Configure and troubleshoot Azure Fabrics, Power Platform components, and associated services.
- Optionally design and implement chatbot solutions using AI Search Index, document chunking, embedding, and Retrieval‑Augmented Generation (RAG) patterns.
- Support ML Ops workflows (AML, MCP servers, AI search) for end‑to‑end machine learning lifecycle.
**Required Skills**
- Azure cloud platform administration & resource provisioning
- Infrastructure as Code: Terraform, ARM templates, Azure CLI
- Networking concepts within Azure (VNet, NSG, firewall, VPN)
- Programming: Python, PySpark, SQL
- Data tools: Azure Data Factory, Databricks, Azure Fabric
- Web/Function Apps development and deployment
- OpenAI model integration in Azure AI Studio
- CI/CD: Jenkins, Git, version control best practices
- Databricks administration and development
- Familiarity with Power Platform (Power Apps, Power Automate)
- Optional: AI Search Index, RAG patterns, chatbot development, ML Ops (AML, MCP servers)
**Required Education & Certifications**
- Bachelor’s degree in Computer Science, Information Technology, or related field (or equivalent experience).
- Azure certifications preferred:
- Microsoft Certified: Azure Administrator Associate
- Microsoft Certified: Azure Data Engineer Associate
- Microsoft Certified: Azure Solutions Architect Expert (optional but advantageous).
- Databricks or PySpark certification highly desirable.
---