- Company Name
- Aaura Softwares
- Job Title
- Lead AWS Data Engineer
- Job Description
-
Job Title: Lead AWS Data Engineer
Role Summary: Lead the design, implementation, and maintenance of enterprise‑grade data platforms on AWS, overseeing large‑scale ELT/ETL pipelines, data lakehouse architecture, and analytical workloads. Drive technical direction, mentor a cross‑functional data engineering team, and translate business requirements into robust data solutions.
Expectations:
- Deliver scalable, production‑ready data pipelines and data products that support analytics, reporting, and ML workloads.
- Lead technical initiatives in a high‑volume, mission‑critical environment.
- Champion cloud best practices, security, governance, and compliance across all data assets.
- Collaborate with stakeholders to prioritize features, estimate effort, and set realistic timelines.
Key Responsibilities:
- Design, build, and optimize ELT/ETL pipelines using AWS services (S3, Glue, Redshift, Athena, EMR/Spark, Lambda, IAM, Lake Formation, Iceberg).
- Implement data lakehouse solutions with Parquet/Apache Iceberg, ensuring high performance and scalability.
- Orchestrate workflows with Airflow or Step Functions and enforce CI/CD pipelines using Terraform/CloudFormation.
- Integrate enterprise systems (Oracle eBusiness, Salesforce, Workday) and on‑prem data sources to AWS.
- Apply data modeling, governance, encryption, key management, and compliance controls across data assets.
- Provide documentation, runbooks, and knowledge transfer to the wider organization.
- Support machine‑learning teams by building pipelines that ingest, transform, and package data for model training and deployment.
Required Skills:
- 12+ years of data engineering experience, or 3 years of this role plus 9+ yrs equivalent.
- 5+ years in a technical or team lead capacity.
- Deep AWS knowledge (S3, Glue, Redshift, Athena, EMR, Lambda, IAM, Lake Formation, Iceberg).
- Proficiency in Python/PySpark or Scala; advanced SQL for warehousing.
- Expertise in workflow orchestration (Airflow, Step Functions) and DevOps practices (CI/CD, automated testing, IaC).
- Strong understanding of data lakehouse architecture, Parquet, and table management.
- Knowledge of data cataloging, metadata management, and lineage tools (Glue Data Catalog, Apache Atlas, Amundsen).
- Experience with data governance, security (encryption, key management), compliance, and regulatory standards.
- Ability to integrate enterprise back‑office applications (Oracle, Salesforce, Workday).
- Familiarity with ML workflow support and data product delivery.
Required Education & Certifications:
- Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or related field.
- 12+ years of hands‑on data engineering experience or equivalent combination of education and experience.
- AWS Certified Data Analytics – Specialty or AWS Solutions Architect certification (preferred).
- Additional cloud platform experience is a plus.