Home
Jobs
United Software Group Inc
Big Data/Hadoop/Splunk Tester

Big Data/Hadoop/Splunk Tester

Remote

Canada

Senior

Freelance

03-12-2025

Share this job:

Score my CV

Skills

SQL GitHub Test Analytics Spark Databricks

Job Specifications

Job Description:

We need a Data QA / Validation Engineer (Contractor) to validate data quality across our Databricks/Spark/Delta Lake pipelines and ensure clean, accurate datasets power downstream analytics, ML, and real-time decisioning systems. This role will create automated data quality tests for ingestion and transformation layers (Bronze → Silver → Gold), enforce schema/data contracts, detect drift, validate feature pipelines, and monitor data anomalies.

Required Qualifications

8+ years of experience in SQL
5+ years of experience with Databricks and Spark
3+ years of experience in data quality frameworks (Great Expectations/Deequ/Soda/dbt tests)
3+ years of experience using GenAI tools (Cursor, Windsurf, GitHub Copilot, Databricks Assistant) to accelerate test generation, documentation, and validation workflows.
5+ years of experience in delivering production-grade validation suites, issue logs, and QA sign-off for critical pipelines.

About the Company

Welcome to USG - Pioneering IT & Engineering solutions provider. Nurturing legacy of 22 years, USG has grown into a global powerhouse, distinguished for its expertise in Information Technology. As a leading IT & Engineering Solutions company, we've cultivated diversified revenue streams and a reputation for excellence. We are an Minority Business Enterprise with focus on diversity and inclusion, proudly serving Global Clients in Banking, Financial, Insurance, Manufacturing, Engineering, Healthcare, Transportation, Communicat... Know more