- Company Name
- Celebal Technologies
- Job Title
- Senior Data Engineer
- Job Description
-
**Job Title**: Senior Data Engineer
**Role Summary**: Design, build, and maintain scalable data infrastructure to enable reliable, high‑quality data access for analytics and business decisions. Collaborate with data scientists, analysts, architects, and cross‑functional teams to deliver robust pipelines, warehousing solutions, and governance practices.
**Expectations**:
- Deliver high‑performance, cost‑efficient data pipelines and warehouses.
- Ensure data accuracy, integrity, and compliance through governance and quality controls.
- Continuously improve systems, documentation, and processes.
**Key Responsibilities**:
1. Design, develop, and maintain scalable data pipelines (ETL/ELT) for large datasets.
2. Integrate data from diverse sources (databases, APIs, third‑party systems) while guaranteeing consistency.
3. Build and support data warehousing solutions for analytics and reporting.
4. Optimize performance, scalability, and cost efficiency of data processing systems.
5. Implement data quality checks, monitoring, and governance practices.
6. Identify and execute improvements to data quality and infrastructure efficiency.
7. Collaborate with data architects, scientists, analysts, and stakeholders to capture requirements.
8. Document data architectures, pipelines, and processes clearly and comprehensively.
9. Coordinate with application, security, infrastructure, and operations teams to resolve design and operational issues.
**Required Skills**:
- Advanced data modeling, database design, and analytics.
- Proficient in Python, Java, or Scala.
- Hands‑on with Apache Spark, Hadoop, or equivalent frameworks.
- Strong SQL expertise; experience with MySQL, Oracle, or SQL Server.
- Experience designing ETL pipelines across multiple operating systems.
- Knowledge of Microsoft Azure data services (e.g., Azure Data Factory, Synapse, Databricks).
- Familiarity with ITIL service delivery, change management, and best practices.
- Understanding of data governance principles.
- Excellent analytical, troubleshooting, and communication skills.
**Required Education & Certifications**:
- Bachelor’s degree in Computer Science, Information Technology, or related field.
- 7+ years in data engineering or a related role.
- Relevant cloud or data engineering certifications (e.g., Microsoft Certified: Azure Data Engineer Associate, Databricks Certified Professional).
**Preferred Qualifications** (optional):
- Knowledge of machine learning concepts, tools, and pipelines.
- Continuous learning mindset to stay current with emerging data engineering trends.