- Company Name
- TXP
- Job Title
- AWS Data Engineer
- Job Description
-
Job Title: AWS Data Engineer
Role Summary:
A cloud‑centric data engineering role within the public sector, focusing on AWS services. The candidate will build, schedule, and maintain batch or micro‑batch ETL jobs, query and transform data stored in S3 or Lakehouse environments, and develop data features on modern analytics platforms.
Expectations:
- Deliver rapid, high‑impact contributions on a short‑turnaround project.
- Maintain operational excellence by troubleshooting and automating data pipeline tasks.
- Possess a valid Senior (SC) Security Clearance.
Key Responsibilities:
- Design, implement, and manage end‑to‑end data pipelines using AWS ETL (e.g., Glue, Lambda) and Azure counterparts.
- Develop and schedule batch/micro‑batch jobs, monitor performance, and resolve runtime issues.
- Query, ingest, and transform data within S3, Azure ADLSv2, or Lakehouse facilities.
- Establish and maintain API‑level and database connectivity for data ingestion and output.
- Version‑control code and infrastructure using GitLab.
- Provision and evolve cloud resources with Terraform or equivalent IaC tools.
- Create data features and transformation procedures on platforms such as AWS LakeFormation, Azure Fabric, Databricks, or Snowflake.
- Automate operational tasks through scripting (Python, Bash, PowerShell, etc.).
Required Skills:
- Programming: Python and other data‑engineering languages (Java, Scala, SQL).
- IDE experience (PyCharm, VS Code, IntelliJ, etc.).
- AWS ETL services (Glue, Athena, Data Pipeline) and Azure ETL services familiarity.
- Knowledge of S3, Azure ADLSv2, and Lakehouse querying.
- API and database integration experience.
- Source control with GitLab and CI/CD pipeline management.
- Terraform (or similar IaC tools).
- Proficiency in modern data platforms: AWS LakeFormation, Azure Fabric, Databricks, Snowflake.
- Scripting for operations automation (Python, Bash, PowerShell).
Required Education & Certifications:
- Bachelor’s degree in Computer Science, Information Technology, or related field (preferred).
- Senior Security Clearance (SC) – must be current and valid.