- Company Name
- Capital One
- Job Title
- Lead Data Engineer
- Job Description
-
**Job Title**
Lead Data Engineer
**Role Summary**
Lead the design, development, and deployment of scalable, cloud‑based data solutions. Coordinate with Agile teams, mentor peers, and drive innovation in data engineering to support enterprise‑wide analytics and machine‑learning initiatives.
**Expectations**
- Deliver high‑quality, production‑ready data pipelines and services.
- Collaborate closely with product managers and cross‑functional teams.
- Maintain rigorous code quality through unit testing, reviews, and performance tuning.
- Stay current with emerging data technologies and share knowledge with the engineering community.
**Key Responsibilities**
- Architect, build, test, and maintain full‑stack data solutions using Java, Scala, and Python.
- Design and manage distributed data workflows (MapReduce, Hadoop, Spark, EMR).
- Implement real‑time streaming and batch processing pipelines.
- Work with relational (e.g., PostgreSQL, MySQL) and NoSQL databases (MongoDB, Cassandra).
- Develop and maintain cloud data warehouses (Redshift, Snowflake).
- Conduct code reviews, unit testing, and performance optimizations.
- Mentor junior engineers and lead technical discussions.
- Apply Agile engineering practices to deliver iterative value.
**Required Skills**
- Programming: Java, Scala, Python, SQL.
- Big Data: Spark, Hadoop, Hive, MapReduce, EMR, Kafka.
- Cloud: AWS, Azure, or GCP data services.
- NoSQL: MongoDB, Cassandra.
- Data Warehousing: Amazon Redshift, Snowflake.
- Real‑time & streaming: Kafka, Spark Streaming, Flink.
- UNIX/Linux: Shell scripting, basic commands.
- Agile methodologies: Scrum or Kanban.
- Strong analytical, problem‑solving, and communication abilities.
**Required Education & Certifications**
- Bachelor’s degree in Computer Science, Engineering, or a related technical field.
- Minimum 4 years of professional application development experience (Java/Scala/Python/SQL).
- Minimum 2 years of experience with big data technologies.
- Minimum 1 year of cloud computing experience (AWS, Azure, or GCP).
- Preferred: 7 + years total dev experience, 4 + years in public cloud, distributed computing, real‑time streaming, NoSQL, and data warehousing, and 2 + years of Agile engineering practice.
---
Newport news, United states
On site
Senior
12-11-2025