Lead Data Engineer

Description:

Noir is seeking a Lead Data Engineer to join a globally recognized technology client. The role involves leading an Agile engineering team, providing technical leadership, coaching, and mentorship. The candidate will be responsible for building a greenfield modern data platform, architecting big data solutions, and developing complex enterprise ETL and ML pipelines.

The ideal candidate should have strong Python, PySpark, SQL skills, a solid understanding of Databricks, and a passion for Data Science, Machine Learning, and AI. Experience with SQL and NoSQL databases, Agile and Scrum methodologies, SOLID principles, continuous integration, and cloud platforms (ideally AWS) is required.

Key Responsibilities:

• Lead client delivery and ensure highest technical standards.

• Collaborate with architects and development teams on design and implementation.

• Create automated tests and promote continuous improvement practices.

• Build a modern data platform using cutting-edge technologies.

• Architect big data solutions and develop complex ETL and ML pipelines.

• Coach, mentor, and guide team members on technical excellence.

Requirements / Skills:

• Python, PySpark, SQL expertise

• Experience with Databricks, R, Machine Learning, AI

• SQL and NoSQL database knowledge (Aurora, MS SQL Server, MySQL)

• Agile and Scrum methodologies, SOLID principles

• CI/CD tools, Infrastructure as Code (Terraform, AWS CDK, AWS CloudFormation, Azure DevOps, Jenkins)

• Cloud platform experience, ideally AWS

Benefits:

• Pension scheme

• Additional company benefits

• Investment in training and career development

• Unlimited career progression for top performers