- Company Name
- Cloud Services
- Job Title
- Middle Data Engineer
- Job Description
-
**Job Title:**
Middle Data Engineer
**Role Summary:**
Design, develop, and maintain end‑to‑end data pipelines on Microsoft Azure, leveraging Spark, Data Lake, Synapse, and other cloud services. Collaborate with global clients to translate business requirements into scalable data architectures and ensure optimal performance and security.
**Expectations:**
- Deliver high‑quality, reusable ETL solutions within agreed timelines.
- Proactively monitor pipeline health and perform performance tuning.
- Communicate effectively with cross‑functional teams and international stakeholders.
- Maintain strong command of English (B1 or higher).
**Key Responsibilities:**
- Build and deploy data ingestion, staging, cleansing, and transformation pipelines using Azure Data Factory, Azure Databricks, Spark, and related services.
- Design and maintain data lake, lakehouse, and data warehouse structures (DWH, Datamart, Delta Lake).
- Implement data quality checks, profiling, and lineage documentation.
- Optimize data models and queries for performance and scalability.
- Secure data assets through access controls, encryption, and compliance practices.
- Collaborate in Agile sprints, using JIRA for task management and continuous delivery.
- Provide technical guidance to teammates and support client projects globally.
**Required Skills:**
- 2+ years of data engineering experience (ideally in a dedicated engineer role).
- Strong knowledge of data modeling, warehousing concepts, and ETL processes.
- Proficiency with Microsoft Azure services: Data Lake Store, Synapse Analytics, Data Factory, Databricks, Cosmos DB, HDInsight, Stream Analytics, Event Grid.
- Distributed computing expertise and familiarity with Apache Spark architecture.
- Advanced SQL skills and experience with data lake/stonehouse patterns (Delta Lake, Lakehouse).
- Understanding of landing, staging, cleansing, profiling, security, and architecture concepts.
- Excellent communication, teamwork, and client‑interaction abilities.
- English language proficiency B1 or higher.
- Experience in Agile development environments (JIRA preferred).
**Required Education & Certifications:**
- Bachelor’s degree in Computer Science, Applied Mathematics, Engineering, or related field.
- Relevant data engineering certification (e.g., Microsoft Certified: Azure Data Engineer Associate) preferred.