Home
Jobs
Compunnel Inc.
Big Data/Hadoop/Splunk Tester -- KUMDC5692693

Big Data/Hadoop/Splunk Tester -- KUMDC5692693

Remote

Halifax, Canada

Senior

Freelance

09-12-2025

Share this job:

Score my CV

Skills

SQL GitHub Test Decision-making Machine Learning Analytics Spark Databricks

Job Specifications

Job Summary

We are seeking a Data QA / Validation Engineer (Contractor) to ensure data quality across Databricks/Spark/Delta Lake pipelines.

The role focuses on validating clean, accurate datasets that power downstream analytics, machine learning, and real-time decision-making systems.

You will design and implement automated data quality tests for ingestion and transformation layers (Bronze ? Silver ? Gold), enforce schema and data contracts, detect drift, validate feature pipelines, and monitor data anomalies.

Key Responsibilities

Develop and execute automated data quality tests for ingestion and transformation layers.
Enforce schema and data contracts across pipelines.
Detect and address data drift and anomalies.
Validate feature pipelines to ensure accuracy and reliability.
Monitor data quality and maintain issue logs for critical pipelines.
Deliver production-grade validation suites and provide QA sign-off for key data processes.

Required Qualifications

8+ years of experience in SQL.
5+ years of experience with Databricks and Spark.
3+ years of experience in data quality frameworks (e.g., Great Expectations, Deequ, Soda, dbt tests).
3+ years of experience using GenAI tools (Cursor, Windsurf, GitHub Copilot, Databricks Assistant) for test generation and validation workflows.
5+ years of experience delivering production-grade validation suites and QA sign-off for critical pipelines.

About the Company

Compunnel, Inc., founded in 1994 and headquartered in New Jersey, is at the forefront of providing customized workforce and digital transformation solutions, enhancing the total experience for our valued business allies. Our approach integrates innovative technologies with a human-centric dialogue, exceeding the needs of our customers, employees, partners, and key growth contributors and delivering exceptional value. With a strategic presence across the United States through over 30 local service delivery centers and glob... Know more