- Company Name
- Viridien
- Job Title
- Data Engineer
- Job Description
-
Job Title: Data Engineer
Role Summary:
Design, build, and maintain robust, secure, and scalable data pipelines and integration solutions that connect internal data platforms to external client systems. Lead the development of modular transformation frameworks and support deployment, monitoring, and performance optimization across the data lifecycle.
Expectations:
- Deliver high‑quality, production‑ready data integrations and transformations on schedule.
- Collaborate with data scientists, analysts, and domain experts to align platform capabilities with business needs.
- Uphold best practices in code quality, security, and performance, continually reducing technical debt.
Key Responsibilities:
- Plan, develop, deploy, and maintain connectors and integrations between the data system and client systems (record, downstream).
- Contribute to architecture and infrastructure, including orchestration, processing logic, and component interactions.
- Build and maintain end‑to‑end, metadata‑driven data pipelines with focus on monitoring, access control, and maintainability.
- Create a modular framework that enables rapid addition of new transformation logic.
- Influence technology choices and produce clear architectural diagrams and documentation for technical and non‑technical stakeholders.
- Partner with end‑users to understand requirements and ensure data accessibility and reliability.
- Share best practices, mentor junior engineers, and align engineering efforts with growth objectives.
Required Skills:
- Proficiency in Python and SQL; ability to write secure, performant code and optimize queries.
- Experience with orchestration and ETL tools (e.g., Airflow).
- Strong RDBMS background (PostgreSQL, Oracle); knowledge of NoSQL (Neo4j, Elastic) and vector databases beneficial.
- Data architecture expertise: data modelling, 3NF, dimensional modelling, medallion architecture.
- Containerization (Docker), version control (Git/GitLab), and CI/CD fundamentals.
- Familiarity with API design (RESTful, GraphQL), networking, and security.
- Proven experience in geoscience, oil & gas, or mining data integration and transformation.
- Ability to communicate complex technical ideas to diverse audiences.
Required Education & Certifications:
- Bachelor’s degree in Computer Science, Software Engineering, or a related discipline.
- No mandatory certifications; cloud or data‑engineering certifications (e.g., Azure Data Engineer Associate) are a plus.