Lead Data Engineer

About the role

This role involves leading the creation of a modern cloud-based data warehouse within the insurance sector, replacing existing legacy reporting systems. The work includes defining technical solutions from business requirements, managing the delivery plan, and identifying potential risks. Candidates will be responsible for ensuring the project's progress and addressing dependencies related to data and legacy systems.

The position combines practical engineering tasks with significant leadership responsibilities. This includes conducting code reviews to maintain quality across SQL, PySpark, and data models, and aligning technical approaches with stakeholders. The lead engineer will also present recommendations and trade-offs to ensure solutions meet business needs and architectural standards.

Success in this role means taking full ownership of the data warehouse delivery, from initial planning to completion. It requires distributing tasks effectively within the team and maintaining a consistent development pace. The lead engineer acts as the primary technical contact for the client, translating their goals into actionable technical plans and ensuring consistent quality throughout the project.

The salary for this position ranges from $149,500 to $241,500 annually.

Skills that matter here

• Azure: This role requires at least two years of experience with Azure, focusing on its data services to build and manage the cloud data warehouse.
• DWH: The core responsibility is to design and build enterprise Data Warehouses, including fact/dimension modeling and aggregation layers.
• SQL: Expert-level SQL skills are necessary for complex analytical queries and performance optimization within the data warehouse.
• PySpark: Strong experience with PySpark is needed for data processing and transformation within the Azure environment.
• Azure Synapse Analytics: This tool is central to the data warehousing solution, requiring strong experience for its implementation and management.
• data modeling: Strong knowledge of data modeling methodologies like Kimball/Inmon, SCD, and historical data handling is essential for designing the data warehouse.

Who this role suits

• A person with a background in data engineering, specifically with at least five years of experience, including two years focused on Azure.
• Someone who thrives in a leadership position, capable of guiding a team through code reviews, architectural decisions, and mentoring.
• An individual who can serve as a primary technical point of contact for clients, translating business needs into technical solutions.
• A professional comfortable working within Scrum teams and managing project delivery from high-level requirements.

From the employer

• Act as a primary technical point of contact for the client, clarifying requirements and translating business goals into technical solutions
• Own and drive the DWH delivery plan, including prioritization, planning, and progress tracking
• Identify risks and dependencies (data, legacy systems, delivery) and proactively propose mitigation strategies
• Perform code reviews and ensure high quality of SQL, PySpark, and data models across the team, ensure consistent quality and architectural alignment
• Align technical solutions with stakeholders and present trade-offs and recommendations
• Distribute tasks within the team and maintain a sustainable development pace.

• 5+ years of experience in Data Engineering, including 2+ years in Azure
• Expert-level SQL skills (complex analytical queries, performance optimization)
• Strong experience with Azure Synapse Analytics, PySpark
• Hands-on experience with Azure Data Lake Storage Gen2 and data layer design (Raw / Silver / Gold)
• Experience with Azure SQL Managed Instance
• Strong knowledge of data modeling (Kimball / Inmon, SCD, historical data handling)
• Solid experience designing and building enterprise Data Warehouses (fact/dimension modeling, aggregation layers)
• Experience in technical leadership (code reviews, architecture decisions, mentoring)
• Experience working in Scrum teams and managing delivery from high-level requirements
• English level B2+ (regular communication with the client).