- Company Name
- CLPS Global
- Job Title
- Big Data Engineer
- Job Description
-
Job title: Big Data Engineer
Role Summary: Design, implement, and optimize large-scale data pipelines and services using AWS and Snowflake, ensuring efficient processing of both batch and API-driven workloads across distributed environments.
Expactations: Minimum 10+ years of end‑to‑end experience in big data architecture, coding, and testing; C2C engagement; expertise in Java, Scala, Python, and Hadoop ecosystem technologies; strong background with AWS EMR, EC2, ECS, S3, Airflow, Step Functions, and API Gateway.
Key Responsibilities:
- Develop and maintain high‑throughput batch and streaming data pipelines in Spark, Hadoop, and MapReduce.
- Build and expose REST and SOAP web services, integrating with Snowflake and other data stores.
- Tune performance of Spark/Hadoop/EMR jobs and Java/Python applications for optimal resource usage.
- Script system automation and data orchestration using Shell and Python.
- Collaborate with cross‑functional teams to translate business requirements into scalable data solutions.
- Troubleshoot and resolve production incidents, ensuring data integrity and system reliability.
- Document architecture, code, and operational procedures for maintainability.
Required Skills:
- Programming: Java, Scala, Python, Kotlin (Java/Scala/Kotlin, Python)
- Big Data Platforms: Snowflake, Hadoop, HDFS, Spark, MapReduce, Cassandra
- Cloud Services: AWS (EMR, EC2, ECS, S3, API Gateway, Step Functions, Airflow)
- Data Orchestration: Airflow, Step Functions
- Performance Tuning: Spark/Hadoop/EMR, Java, Python
- Shell/Scripting: Bash, Python
- Microservices: REST API, SOAP web services
Required Education & Certifications:
- Bachelor’s degree in Computer Science, Information Technology, or related field (preferred).
- AWS Certified Solutions Architect – Associate or equivalent cloud certification (preferred).
California, United states
On site
Senior
09-03-2026