Skills

Communication Python Unity SQL Data Engineering CI/CD DevOps Problem-solving Architecture Azure AWS Analytics CI/CD Pipelines Databricks ETL Processes

Job Specifications

Job Title: Azure Developer (Databricks)

Location : Halifax, NX Canada (100% Permanent remote work accepted from anywhere in US; however we’d prefer EST and CST based resources)

Duration : 12+ Months Contract

Teams Meeting Interview

Job Description:

Note: Please look for senior candidates (10+ years at least). Experience in cybersecurity and/or healthcare data environments is highly preferred.

We are seeking a highly skilled Senior Data Engineer with deep expertise in Databricks and Python, and substantial experience implementing medallion architecture. The ideal candidate will play a key role in designing and maintaining scalable, secure, and reliable data platforms that support advanced analytics and business insights. Experience in cybersecurity and/or healthcare data environments is highly preferred.

Key Responsibilities:

Design, implement, and optimize robust data pipelines and ETL processes using Databricks and Python.
Architect and manage data solutions leveraging the medallion architecture (Bronze, Silver, Gold layers) for scalable and efficient data processing.
Ensure data quality, integrity, and compliance with security and privacy regulations, particularly in healthcare or cybersecurity contexts.
Collaborate with data scientists, analysts, and cross-functional teams to deliver high-quality, secure data solutions.
Develop and maintain data lakehouse architectures, utilizing Delta Lake and Unity Catalog.
Monitor, troubleshoot, and optimize data workflows in production environments.
Contribute to the development and enforcement of data engineering standards, especially with respect to secure data handling.

Required Qualifications:

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
5+ years of experience in data engineering roles, with a strong emphasis on Databricks and Python.
Proven track record implementing medallion architecture in cloud-based environments.
Advanced proficiency in SQL, data modeling, and ETL frameworks.
Excellent problem-solving, communication, and collaboration skills.

Preferred Qualifications:

Experience with Azure Databricks or AWS Databricks.
Familiarity with CI/CD pipelines, DevOps, and deployment automation.
Knowledge of regulatory requirements such as HIPAA, HITECH, or NIST frameworks.
Exposure to data security best practices and secure cloud architectures.
Experience working with sensitive data in cybersecurity or healthcare, and familiarity with relevant compliance standards.

About the Company

Welcome to USG - Pioneering IT & Engineering solutions provider. Nurturing legacy of 22 years, USG has grown into a global powerhouse, distinguished for its expertise in Information Technology. As a leading IT & Engineering Solutions company, we've cultivated diversified revenue streams and a reputation for excellence. We are an Minority Business Enterprise with focus on diversity and inclusion, proudly serving Global Clients in Banking, Financial, Insurance, Manufacturing, Engineering, Healthcare, Transportation, Communicat... Know more