- Company Name
- PRI Global
- Job Title
- Senior Data Engineer
- Job Description
-
Job Title: Senior Data Engineer
Role Summary:
Seasoned data engineering professional focused on building and optimizing end‑to‑end data solutions in Databricks. Responsible for designing scalable data warehouses, orchestrating complex workflows, and delivering production‑ready pipelines and ML applications across cloud platforms.
Expectations:
- Deliver production‑ready data solutions on time.
- Communicate project issues and concerns promptly.
- Collaborate effectively within Agile, cross‑functional, geographically distributed teams.
- Maintain high standards of data quality, performance, and security.
Key Responsibilities:
- Build, test, and deploy data pipelines using Databricks (Spark, Delta Lake, Unity Catalog).
- Design data models and warehouse architectures for Big Data ecosystems.
- Develop and manage ETL/ELT workflows with orchestration tools (Apache Airflow, NiFi, LakeHouse Connect).
- Optimize database schemas, SQL queries, and ETL scripts for performance.
- Create and deploy scalable data‑driven or ML applications (feature engineering, model deployment, monitoring).
- Integrate solutions with cloud services (Azure, AWS, GCP).
- Troubleshoot and resolve complex data challenges.
- Collaborate with product, data science, and infrastructure teams to align on requirements and deliverables.
Required Skills:
- Databricks (Databricks Runtime, Delta Lake, Unity Catalog)
- Programming: Python, Java, Scala (backend development background)
- Data engineering & modeling in Big Data environments
- Workflow orchestration: Apache Airflow, NiFi, LakeHouse Connect
- Spark (PySpark, Spark SQL)
- Performance tuning for schemas, SQL, and ETL processes
- Cloud services: Azure, AWS, GCP
- Agile development practices
- Strong analytical, problem‑solving, and communication abilities
Required Education & Certifications:
- Bachelor’s degree in Computer Science, Software Engineering, or related field (equivalent practical experience acceptable).
- Relevant certifications (e.g., Databricks Certified Professional Data Engineer, Azure/AWS Big Data) are a plus but not mandatory.