Staff Data Engineer

Salary: £? - ? per year

Requirements:

• We are looking for experience in Staff, Principal or Lead Data Engineering roles.
• We need strong Python skills and expert-level PySpark experience.
• We require experience with AWS Glue, Apache Spark and Apache Airflow.
• We are looking for experience with cloud-native Data Engineering architectures.
• We need experience delivering large-scale data pipeline modernisation.
• We value experience building reusable engineering patterns within enterprise environments.
• We require experience in data testing, validation and observability.
• We are looking for experience using AI-assisted software development tools and modern coding assistants.
• Experience working within regulated or highly governed organisations is important.

Responsibilities:

• We are defining technical standards and engineering patterns for a large-scale Data Modernisation programme.
• We are re-engineering complex legacy data pipelines into production-grade PySpark.
• We are building scalable, reusable data movement frameworks across multiple engineering teams.
• We are driving best practice around AWS Glue, Apache Spark and Apache Airflow.
• We are designing high-quality testing frameworks covering performance, data quality and reliability.
• We are leveraging Agentic AI development tools to improve engineering productivity while maintaining code quality.
• We are coaching and influencing experienced engineers through technical leadership rather than line management.
• We are working closely with enterprise architecture and platform teams to improve how data is governed, managed and consumed.

Technologies:

• AI
• Airflow
• AWS
• AWS Glue
• Cloud
• Python
• PySpark
• Spark

More:

We are supporting a global, technology-led organisation undertaking a large-scale cloud data transformation programme. This is a Staff-level contract opportunity for an engineer who can combine deep hands-on technical expertise with organisation-wide technical influence across multiple engineering teams. The role focuses on enterprise-scale data movement, cloud-native architectures and AI-assisted software engineering within a highly collaborative engineering culture. The contract is initial 6 months, inside IR35, based in London with 3 days per week on-site.

last updated 31 week of 2026