Data Engineering Lead AWS & AI - Manchester

Salary: £95,000 - 95,000 per year

Requirements:

• Experienced Data Engineer or Data Engineering Lead with experience building and operating complete data platforms
• Experience with batch processing, data warehousing, ELT and orchestration
• Experience with modern streaming technologies such as Flink or comparable platforms
• Production AWS data engineering experience, including services such as S3, Lambda, Glue, Athena, Kinesis, RDS, IAM and CloudWatch
• Experience with lakehouse architectures, open table formats such as Iceberg, medallion-style data layering and dimensional modelling
• Strong SQL and Python development skills
• Experience with infrastructure as code using Terraform, CI/CD, automated testing and modern engineering practices
• Experience with SQL Server, SSIS/SSDT and Power BI
• Experience with semantic models, shared metrics and data glossaries
• Experience with data governance, retention, lineage, classification and auditability
• Experience managing, mentoring or developing Data Engineers
• Confidence making architectural decisions, leading change and assessing when existing technology should be evolved or replaced
• Ability to use advanced AI coding assistants, such as Claude Code or equivalent tools, while maintaining appropriate quality and security controls
• Desirable: experience in financial services or another regulated environment
• Desirable: PySpark experience for large-scale batch or streaming transformations
• Desirable: experience migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures
• Desirable: experience with Customer Data Platforms or event-driven systems
• Desirable: experience with data-quality tools such as dbt tests or Great Expectations
• Desirable: experience with MCP servers or governed natural-language data interfaces
• Desirable: experience with financial crime, fraud or regulatory-reporting data
• Desirable: Power BI administration and semantic modelling experience

Responsibilities:

• Own the reliability, security and evolution of our data platform
• Modernise our existing SQL Server, SSIS/SSDT and Power BI environment
• Develop scalable batch and real-time data-processing capabilities
• Lead the design of a Customer Data Platform
• Build and evolve a modern AWS lakehouse architecture
• Create governed semantic models, consistent metrics and clear data definitions
• Enable secure, self-service data analysis through modern AI tools
• Automate manual, repetitive and fragile data processes
• Support BI, Data Science, Machine Learning and Applied AI teams through a unified platform
• Own data governance, platform security, monitoring and operating costs
• Manage, develop and help grow our Data Engineering team
• Work closely with technical leaders and senior stakeholders across our organisation

Technologies:

• AI
• AWS
• Lambda
• Business Intelligence
• CI/CD
• Claude Code
• CloudWatch
• Flink
• IAM
• Support
• Machine Learning
• MCP
• Power BI
• Python
• PySpark
• SQL
• Security
• SSIS
• Terraform
• dbt
• Cloud

More:

We are an established financial services organisation seeking a hands-on Data Engineering Lead to take ownership of our existing data estate and deliver a modern platform supporting Business Intelligence, Data Science and Applied AI. This influential role combines technical leadership, platform ownership and line management. You will lead a small team while remaining involved in architecture, technology selection and engineering delivery. You will have the opportunity to shape how we use data and AI, introduce modern technologies, develop our engineering team and build a platform for future analytics, AI products and strategic decision-making. The role is based in Manchester with a hybrid working pattern.

last updated 39 week of 2026