- Company Name
- Profectus IT
- Job Title
- Azure Data Engineer
- Job Description
-
**Job Title**
Azure Data Engineer
**Role Summary**
Design, develop, and maintain secure, scalable data pipelines on Microsoft Azure to provide reliable data for Data Science, BI, and business teams. Lead modern data architecture implementation (Data Lake, Delta Lake, Lakehouse) and support self‑service reporting via Power BI and Databricks.
**Expectations**
* Deliver high‑quality data solutions, ensuring performance, reliability, and security.
* Collaborate cross‑functionally with analysts, scientists, and stakeholders to translate business needs into technical requirements.
* Adhere to CI/CD best practices, documentation standards, and governance policies.
* Communicate technical concepts clearly to both technical and non‑technical audiences.
**Key Responsibilities**
1. Design and develop data pipelines using Azure Data Factory, Azure Databricks, or equivalent services.
2. Implement and maintain Data Lake, Delta Lake, and Lakehouse architectures.
3. Automate data ingestion, transformation, and exposure workflows.
4. Partner with Data Analysts, Data Scientists, and business stakeholders to understand requirements and recommend solutions.
5. Document data flows, models, and best‑practice guidelines.
6. Assist in implementing self‑service Power BI reporting and Databricks notebooks.
7. Manage Azure platform components (SQL, Storage, Key Vault, DevOps) and monitor performance.
8. Apply real‑time data streaming and integration patterns (e.g., REST, AMQP) as needed.
**Required Skills**
* Proficiency with Azure services: Data Factory, Databricks, SQL, Storage, Key Vault, DevOps.
* Experience with distributed processing frameworks (Spark, PySpark, Databricks).
* Strong SQL, Python, and/or Scala programming.
* Knowledge of CI/CD principles applied to data engineering.
* Power BI administration and governance.
* Real‑time data streaming basics.
* Application integration patterns (REST API, AMQP).
* Adherence to data engineering best practices.
**Required Education & Certifications**
* Bachelor’s degree in Computer Science, Data Engineering, or related field.
* Minimum 5 years of experience as a data engineer (Azure experience required).
* Microsoft Azure certification (DP‑203) and/or Databricks Data Engineer Associate certification preferred.
**Nice to Have**
* Agile/Scrum experience.
* Monitoring tools (Azure Monitor, Log Analytics).
* SAP S/4HANA knowledge as a data source.
* Azure architecture design and governance experience.