- Company Name
- TechTriad
- Job Title
- Senior AWS Data Engineer
- Job Description
-
Job title: Senior AWS Data Engineer
Role Summary: Design, build, and optimize scalable data pipelines, feature stores, and machine learning workflows using Databricks and AWS services to enable analytics, personalization, and performance insight applications.
Expactations: • Deploy production-ready data solutions for batch and real‑time processing • Scale data infrastructure to support ML model training and inference • Ensure data quality, security, and compliance • Collaborate with data scientists, ML engineers, and product teams • Demonstrate strong coding, optimization, and troubleshooting skills
Key Responsibilities:
• Architect and implement data pipelines in Databricks (PySpark, Delta Lake, SQL) and AWS (S3, Glue, EMR, Kinesis, Lambda)
• Build and maintain feature stores (Databricks Feature Store, Feast) for ML models
• Develop and maintain batch and streaming workflows, integrating with MLflow and SageMaker
• Apply CI/CD practices to data pipelines and infrastructure-as-code (e.g., Terraform, CloudFormation)
• Design and enforce data quality frameworks and monitoring
• Optimize performance and cost of distributed data processing jobs
• Document architecture, processes, and operational guidelines
• Mentor junior engineers and provide technical leadership
Required Skills:
• 7–10+ years of data engineering or distributed systems experience
• Deep expertise in Databricks (PySpark, Delta Lake, SQL) and AWS services (S3, Glue, EMR, Kinesis, Lambda)
• Proven experience with feature store design and implementation
• Strong proficiency in Python (preferred) and familiarity with Scala/Java
• Hands‑on experience with batch and streaming data pipelines, large‑scale distributed computing, and ML model pipelines (MLflow, SageMaker)
• Knowledge of CI/CD, data quality frameworks, and infrastructure-as-code
• Excellent communication and problem‑solving skills
• Ability to work collaboratively in cross‑functional teams
Required Education & Certifications:
• Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience)
• AWS certifications (e.g., AWS Certified Data Analytics – Specialty, AWS Certified DevOps Engineer) are a plus, but not mandatory if equivalent experience exists