- Company Name
- CloudTech Innovations
- Job Title
- Databricks Solution Architect
- Job Description
-
**Job Title**
Databricks Solution Architect
**Role Summary**
Architect, design, and implement end‑to‑end, cloud‑native data platforms that enable scalable analytics and AI workflows using Databricks, Delta Lake, MLflow, and supporting cloud services. Lead cross‑functional collaboration to translate business objectives into robust, governed, and secure data solutions while driving performance and cost optimization.
**Expectations**
- 10+ years in enterprise software or data architecture roles, with deep expertise in Databricks, Apache Spark, and Delta Lake.
- Proven track record designing large‑scale ETL/ELT pipelines, data lakes, lakehouses, and real‑time/streaming data solutions.
- Strong command of AWS and GCP, including S3, ADLS, BigQuery, or Redshift, and experience with Glue, Data Factory, and other cloud data services.
- Demonstrated ability to implement data governance, security (IAM, encryption, Unity Catalog), and data quality controls.
- Effective communication and stakeholder management skills across engineering, data science, and business teams.
**Key Responsibilities**
- Design and lead the deployment of cloud‑native data platforms (Databricks, Delta Lake, MLflow).
- Define architectures for large‑scale ETL/ELT pipelines, data lakes, and real‑time or streaming data solutions.
- Collaborate with data engineers, data scientists, and stakeholders to translate business goals into technical architectures.
- Integrate Databricks notebooks, Spark, and cloud‑native services (AWS Glue, Azure Data Factory) for batch and real‑time processing.
- Implement governance, security, and compliance using Unity Catalog, IAM, encryption, and data quality frameworks.
- Define integration patterns via REST APIs, event‑driven messaging (Kafka, Pub/Sub), and distributed systems design.
- Participate in architectural reviews, performance tuning, and cost optimization across distributed compute frameworks.
- Stay current on emerging tools and technologies in data architecture, cloud infrastructure, and ML Ops.
**Required Skills**
- Databricks, Apache Spark, Delta Lake, MLflow expertise.
- Cloud platform proficiency (AWS, GCP) with S3, ADLS, BigQuery, Redshift, Glue, Data Factory.
- Streaming platforms: Kafka, Kinesis, Pub/Sub.
- Data modeling, governance, and orchestration (Airflow, dbt, or equivalent).
- Performance optimization, data security best practices, cloud cost management.
- Strong communication and stakeholder management.
**Required Education & Certifications**
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
- 10+ years of enterprise software or data architecture experience.
- Preferred certifications: Databricks Certified Professional, AWS/Azure/GCP Solution Architect, TOGAF.
---