- Company Name
- Brio Digital | Certified B Corp
- Job Title
- Data Engineer
- Job Description
-
**Job Title**: Data Engineer
**Role Summary**
Design and maintain end‑to‑end data pipelines within Microsoft Azure, leveraging Databricks, Python, and SQL to transform, cleanse, and load healthcare data from sources such as MongoDB. Ensure data quality, governance, and scalability in a cloud‑native architecture, collaborating closely with analysts, scientists, and delivery teams.
**Expectations**
- Deliver reliable, maintainable ETL solutions for both analytical and operational workloads.
- Demonstrate expertise in Azure data services and large‑scale data processing.
- Apply modern data architecture and governance best practices.
- Communicate effectively with stakeholders and support agile project delivery.
**Key Responsibilities**
- Design, develop, and maintain Azure-based data pipelines (Data Factory, Synapse, Data Lake, Databricks).
- Build and optimize ETL processes in Python, SQL, and Spark.
- Integrate data from diverse sources, including MongoDB and other NoSQL databases.
- Implement data quality, governance, validation, and testing frameworks.
- Collaborate with data analysts, scientists, and delivery teams to ensure data accessibility and trust.
- Contribute to the evolution of scalable, cloud‑native data architectures.
- Participate in CI/CD, version control, and Agile workflows.
**Required Skills**
- Strong Python programming for ETL and data engineering.
- Advanced SQL: data modelling, optimisation, stored procedures.
- Proficiency with Azure services: Data Factory, Synapse, Data Lake, Databricks.
- Experience with NoSQL databases (e.g., MongoDB).
- Knowledge of data architecture, data governance, and data quality practices.
- Familiarity with CI/CD pipelines, version control, and Agile delivery.
- Problem‑solving and stakeholder communication skills.
- NHS or Central Government data experience (essential).
**Required Education & Certifications**
- Bachelor’s degree (or equivalent) in Computer Science, Information Technology, Data Engineering, or related field.
- Industry certifications such as Microsoft Certified: Azure Data Engineer Associate preferred.
- Understanding of healthcare data standards (e.g., FHIR) a plus.