Job Specifications
Investment Management Industry
Senior Data Engineer
Type: FTE, Permanent
Location: Downtown Toronto (2-3 day/week in office)
# of Openings: 1
Salary: $115k-$130k, depending on years of experience (+bonus, full benefits)
Why do you have this need? Big growth curve - want to expand to 5-8 Data Engineers. Want people who know what they’re doing, can come in and get to work. The two intermediate data engineers, they’re okay to be a little more cloud focused, but the senior engineer must have both cloud and on-prem
What are they going to accomplish on a high level? Design, deploy, manage cloud environment for different teams and stakeholders who have differing and various requirements for different projects. Responsible for giving them the proper environment for their applications.
Day to day what are the responsibilities:
Creating ETL processes
PH has legacy systems and Databases, would need to maintain them
Migrate those SQL server to other servers - upcoming work
Write auto scripts using Python
Work is purely creating data pipeline, person will mostly work on Azure cloud, sql, data factory
Large amount of data, and not going to move right away, but have ongoing projects that are cloud based.
There are 5 SQL server, Salesforce, API as well.
Validating the data. Once they create pipeline, have to validate and test pipeline
Train quantitative analysts at PH to consume data and data storage efficiently. They currently grab whole data base, play with it in R, then dump back in database. Data team itself is very good at dealing with data, moving it around, being efficient with storage, so they need to teach others outside of team to be better.
PowerBI and Analytics - Responsible for creating diff visualization for the org, write power queries
May be assigned to work separate projects
Cross-team work. Will have BA team that will speak to stakeholder, but once in awhile will communicate directly with them
Must-haves/key skills needed
Must have on-prem experience, along with cloud
Senior person will be working more closely with the Data Architect - experience doing that is key
6-7 years: Azure, Snowflake and Databricks, SQL and Python- comfortable coding in Python, PySpark- using for Databricks and Snowflake
Python, PySpark, Azure Data Factory, SQL Database, Glove storage, Snowflake, Azure Cloud Services
ETL tools: Data Factory, Data Bricks, PySpark
Object oriented - C# and Python
Big data - Hadoop spark
Knowledge on PowerBI and Analytics
Engineer is going to be working on data, working in conjunction with the data architect on the single pane of glass (may be called data lake or data warehouse). Will be working on that and be helping support data flows for their order management system, for example in particular, and also working on data flows for their second engine division.
A lot of spread and support for a lot of moving parts
Working closely with Picton’s quant team as well. Their quant team is very, very good at quantitative analytics and very, very bad at pulling information from databases. The CIO really would like this Senior Engineer to teach them how to do it. They don't have to had had mentoring. If they do, that's a big bonus though, if they've worked with other teams, in particular developers, to show them how to retrieve data and how to use data properly. If candidate has this, amongst other must-haves, would be top of CIO’s list.
Plusses/education/certifications
Cosmos Database (good to have)
Certifications
Kafka - not using at moment but that will be introduced later for Real time Data
About the Company
In our relentless pursuit of greatness, we are dedicated to developing individuals, creating exceptional teams, and cultivating a unique culture of unity and care. As providers of digital talent solutions, we aim to positively impact businesses and communities globally. We would be honored to be your trusted and uncommon partner on this journey.
Know more