Home
Jobs
Andiamo
Big Data Software Engineer - Intelligent DataOps Software Leader

Big Data Software Engineer - Intelligent DataOps Software Leader

On site

San francisco, United states

Full Time

21-12-2025

Share this job:

Score my CV

Skills

Go SQL Big Data Snowflake Spark

Job Specifications

Big Data Software Engineer - Intelligent DataOps Software Leader
Big Data Software Engineer - Batch & OLAP
About The Role
Build robust batch pipelines and analytical stores for large-scale reporting, experimentation, and ML feature generation.
Responsibilities
Develop cost-efficient batch processing (Spark/Dataproc/EMR) and orchestration (Airflow/Dagster).
Design lakehouse tables (Iceberg/Delta/Hudi) with compaction, partitioning, and Z-ordering.
Implement semantic/metrics layers and OLAP acceleration (BigQuery/Snowflake/ClickHouse).
Enforce governance: lineage, access controls, PII handling, and retention policies.
Requirements
4+ years in big data; expert SQL and performance tuning.
Strong understanding of storage formats, file layout, and query engines.
About Andiamo
Talent Partners for the AI Revolution. As a globally recognized staffing and consulting firm, we specialize in placing the top 2% of technology and go-to-market professionals with the world's largest and most well-known companies. For over 20 years, we've maintained the status of tier-one vendor for firms such as Palantir, Amazon, Fluidstack, Bloomberg, Relativity Space, Firefly, MasterCard, Visa, Two Sigma, Citadel, as well as other major financial services firms, elite hedge funds, Google-backed tech start-ups, and major software firms. Our talent solutions include Permanent Placement, Contract Staffing, Executive Search, and Dedicated Recruiting Services (RPO).
Seniority level: Mid-Senior level
Employment type: Full-time
Job function: Engineering and Information Technology; Industries: Staffing and Recruiting

About the Company

We're a different kind of recruiting firm. We employ Research & Data Analysts alongside Recruiters to bring our clients the top 2% of passive technology and go-to-market talent. With unique data partnerships and a revolutionary technology platform, we're mining and curating massive amounts of data to bridge the talent gap. Our data-driven approach has helped us become the top recruiting partners of Amazon.com, HBO, Bloomberg, Goldman Sachs, TripAdvisor, Audible, MasterCard, and others. Andiamo has locations in New York, ... Know more