- Company Name
- W.G. CONSULTING
- Job Title
- Data Engineer – Plateforme Data Cloud F/H
- Job Description
-
**Job title:** Data Engineer – Cloud Data Platform
**Role Summary:**
Design, build, and maintain scalable data pipelines and data infrastructure on Google Cloud Platform (GCP). Collaborate with product, data stewardship, and engineering teams to centralize heterogeneous data sources, ensure data governance, security, and quality, and enable self‑service analytics.
**Expectations:**
- Deliver robust, modular data solutions in an Agile environment.
- Actively participate in all project phases: design, planning, development, testing, deployment, and monitoring.
- Demonstrate deep knowledge of data architecture, cloud services, and best‑practice frameworks (CI/CD, documentation, security).
**Key Responsibilities:**
1. Design, develop, and maintain scalable data pipelines using Spark, Airflow, and GCP services (BigQuery, Dataproc, Pub/Sub, Cloud Storage, Dataplex, Delta Lake).
2. Structure and optimize data storage and cataloging, ensuring performant, secure, and compliant data assets.
3. Build and sustain CI/CD workflows with GitLab CI/CD, Terraform, and Terragrunt.
4. Collaborate with stakeholders to translate business requirements into technical specifications and support data exploration and query usage.
5. Ensure system stability, performance, and security; apply data governance practices and monitor compliance.
6. Contribute to technical documentation, knowledge sharing, and continuous learning of emerging data technologies.
**Required Skills:**
- Python, SQL, Apache Spark, Apache Airflow
- Google Cloud Platform: BigQuery, Dataproc, Pub/Sub, Cloud Storage, Dataplex, Delta Lake
- Infrastructure as Code: Terraform, Terragrunt
- Version control and CI/CD: GitLab, GitLab CI/CD
- Data architecture and data structures
- Data governance, security, and quality principles
- Agile methodologies (Scrum or Kanban)
- Ability to simplify complex technical concepts for non‑technical audiences
**Required Education & Certifications:**
- Bachelor’s degree in Computer Science, Data Engineering, or related field (or equivalent professional experience).
- GCP certifications (e.g., Google Cloud Professional Data Engineer) are advantageous.