Job Specifications
Greymatter Innovationz helps you stay digitally relevant across domains, technologies, and skillsets, every day.
We are looking for:
Lead AWS Cloud Engineer - Databricks Platform
Location: Princeton, NJ (Hybrid - 3 days onsite)
Duration: Contract Or FTE
Position Overview:
We are seeking a highly skilled and experienced AWS Cloud Engineer with a strong background in Databricks to join our dynamic cloud engineering team. The ideal candidate will have extensive experience in designing, deploying, and managing cloud infrastructure on AWS, along with in-depth expertise in the Databricks platform for data engineering, analytics, and machine learning workloads. The candidate should also have prior leadership experience, as they will be responsible for guiding a team of engineers in cloud-related initiatives, best practices, and ensuring the successful delivery of data-driven solutions.
Key Responsibilities:
Cloud Infrastructure Design & Implementation:
Lead the design, implementation, and optimization of scalable and secure AWS cloud solutions.
Utilize AWS services such as EC2, S3, Lambda, RDS, VPC, IAM, and others for building robust cloud-based architectures.
Work closely with the architecture team to define infrastructure requirements and recommend suitable AWS technologies.
Databricks Platform Expertise:
Leverage Databricks for building and managing data pipelines, running Spark-based analytics workloads, and orchestrating machine learning models.
Design, optimize, and maintain Databricks clusters and notebooks for data processing, transformation, and analysis.
Integrate Databricks with other AWS services (e.g., S3, Redshift, Glue) to build comprehensive end-to-end data solutions.
Provide leadership in best practices for working with Databricks to ensure reliability, scalability, and efficiency of data workflows.
Leadership & Mentoring:
Lead a team of cloud engineers, guiding them in implementing solutions that align with organizational goals and industry best practices.
Mentor junior team members and provide technical guidance on cloud-related projects, Databricks integration, and infrastructure optimization.
Facilitate cross-functional collaboration between engineering, data science, and business teams.
Automation & Optimization:
Automate cloud infrastructure provisioning and management using Infrastructure-as-Code tools (e.g., Terraform, CloudFormation).
Continuously monitor and optimize AWS resources to improve performance and reduce costs.
Troubleshooting & Support:
Provide expert-level troubleshooting and support for cloud infrastructure, Databricks platform issues, and production systems.
Identify and resolve performance bottlenecks, security risks, and system inefficiencies across AWS and Databricks environments.
Documentation & Reporting:
Document cloud infrastructure, Databricks configurations, and solutions for internal knowledge sharing.
Prepare regular reports on system performance, cost management, and the status of cloud initiatives.
Key Requirements:
Experience:
5+ years of experience in AWS cloud engineering, with a proven track record of delivering cloud-based solutions.
3+ years of hands-on experience with Databricks platform, including building and managing data pipelines, notebooks, and Spark-based workloads.
2+ years of experience leading or mentoring a team of engineers, with strong leadership and team-building skills.
Technical Skills:
Proficient in AWS cloud services (EC2, S3, Lambda, RDS, VPC, IAM, CloudWatch, etc.).
Strong knowledge of Databricks, Apache Spark, and data engineering best practices.
Experience with containerization and orchestration tools (Docker, Kubernetes).
Familiar with Infrastructure-as-Code tools (Terraform, CloudFormation).
Experience with data processing frameworks, data lakes, and analytics pipelines.
Additional Skills:
Strong understanding of security best practices in cloud environments.
Ability to manage complex cloud projects with multiple stakeholders.
Excellent communication skills, with the ability to explain technical concepts to non-technical stakeholders.
Experience with Agile methodologies is a plus.
Education & Certifications:
Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field (or equivalent experience).
AWS Certified Solutions Architect - Professional (preferred).
Databricks Certified Associate Developer (preferred, but not required).
At Greymatter Innovationz,
We offer:
Motivating Work Environment.
Excellent Work Culture.
Help you to upgrade yourself to the next level.
And More!!!