Job Specifications
Databricks Technical Lead (Azure) | Commodity Trading | Greenfield Data Platform | Up to £1,000 per day Inside IR35 | 6-month rolling | Hybrid 2–3 days London
A Databricks Technical Lead is required for a large, complex commodity trading organisation undertaking a greenfield implementation of Databricks on Azure. This is a strategic data platform programme, focused on building a modern, scalable lakehouse architecture to support analytics, reporting, and downstream data use cases.
This Databricks Technical Lead role will take end-to-end technical ownership of the Databricks platform, leading design decisions, setting engineering standards, and guiding delivery across a hybrid on-prem and Azure environment.
You will act as the senior technical authority, working closely with data engineers, architects, platform teams, and senior stakeholders to ensure the platform is built correctly from day one.
Key Responsibilities of the Databricks Technical Lead:
Lead the greenfield design and implementation of Databricks on Azure.
Define and own platform architecture, patterns, and best practices.
Design and oversee development of scalable data pipelines using Spark, Python, and SQL.
Lead ingestion from on-prem data sources into Azure and Databricks.
Drive adoption of lakehouse architecture, data modelling standards, and performance optimisation.
Act as the technical escalation point for Databricks, Spark, and Azure data services.
Provide technical leadership and mentorship to data engineers.
Conduct design reviews and ensure high engineering quality.
Work with security and platform teams on governance, access control, and compliance.
Support early POCs, MVPs, and longer-term platform evolution.
6-month rolling contract (long-term programme)
Up to £1,000 per day – Inside IR35
Hybrid working: 2–3 days per week, City of London
Technology Environment
Databricks (Spark, Jobs, Notebooks, Lakehouse)
Azure
Hybrid on-prem / cloud data architecture
Essential Skills & Experience for the Databricks Technical Lead:
Proven experience as a Databricks Technical Lead or Senior Data Engineer / Architect
Strong hands-on Databricks experience on Azure
Experience leading greenfield data platform implementations
Strong Python and SQL skills for data engineering
Experience integrating on-prem systems with Azure
Experience defining engineering standards and architecture
Strong stakeholder communication and leadership skills
Domain experience (e.g. energy, commodities, finance) is NOT required — this role is focused on technical delivery and platform leadership.
If you are interested, reply with your latest CV or email me at Oscar@virtuetech.io
About the Company
From start-ups to unicorns, you need great people to grow, so at VirtueTech we do recruitment differently.
We believe companies are defined by their people, so connecting clients and candidates through mu-tual values is key for a thriving business. With a joint brand mission being paramount for success, you need an agency aligned with that purpose. Which is precisely where we come in...
At VirtueTech jobs are more than just a salary, they're a passion. So we understand and reflect our clients values, in both ourselves and...
Know more