Senior/Lead Data Engineer

Salary: £? - ? per year

Requirements:

• Strong Spark and PySpark engineering experience
• Experience developing and modernising large-scale data pipelines
• Strong Python skills and software engineering fundamentals
• Experience with performance, scalability, and workload optimisation
• Cloud data engineering experience, ideally on AWS
• Experience with CI/CD and modern engineering practices
• Experience working in complex, enterprise-scale data environments
• Senior or Lead-level data engineering experience
• Ability to use AI tools to accelerate development while maintaining strong engineering judgement

Responsibilities:

• Build, modernise, and optimise data workloads in a large-scale, complex environment
• Work hands-on across engineering rather than overseeing from a distance
• Develop and modernise large-scale data pipelines
• Apply performance and scalability improvements to data workloads
• Use AI to support the development lifecycle while reviewing and correcting AI-generated code where needed
• Deliver robust engineering solutions that scale properly

Technologies:

• AI
• AWS
• CI/CD
• Cloud
• Support
• Python
• PySpark
• Spark

More:

We are a major UK organisation making significant investment across our Data and AI estate. This is a Senior or Lead Data Engineer contract role based in London on a hybrid arrangement with three days onsite. The initial contract duration is six months, and the role is inside IR35. We are looking for a hands-on engineer who enjoys tackling genuinely large-scale data challenges and working in a modern, complex enterprise environment.

last updated 33 week of 2026