Senior Manager - Principal Data Engineer

Salary: £41,000 - 81,000 per year

Requirements:

• Extensive experience in data engineering, with advanced hands-on expertise in Databricks, including PySpark, Delta Lake, and Unity Catalog
• Extensive experience designing advanced data models, including medallion architectures, data lakes, and enterprise data warehouses
• Expert-level proficiency in SQL for complex data transformation, optimization, and analytical workload development
• Advanced proficiency in Python for data engineering, pipeline automation, and integration development
• Deep expertise in PySpark for data processing and transformation
• Proven mastery of cloud platforms, with a strong preference for Microsoft Azure
• Expertise in ETL/ELT development and data pipeline orchestration, such as Databricks Workflows, Delta Live Tables, and Azure Data Factory
• Extensive experience with Git version control, GitHub collaboration workflows, and modern CI/CD practices, including automated testing and deployment pipeline automation for data engineering workloads
• Hands-on expertise with Agile methodologies such as Scrum and Kanban, and DevOps practices applied to data engineering contexts
• Demonstrated capability to balance technical execution with people management responsibilities and product accountability
• Proven analytical thinking and advanced problem-solving skills with the ability to tackle complex, enterprise-scale technical challenges
• Strong decision-making capabilities with the ability to evaluate technical trade-offs and make pragmatic choices under pressure and ambiguity
• Strong track record in managing and coaching engineering teams, with a minimum of 3+ years in Tech Lead roles
• Experience with AI-native software delivery and specification-driven development, including AI-assisted engineering tools, automated quality assurance, modern testing practices, and highly automated CI/CD workflows within enterprise-scale environments
• Relevant associate-level or expert-level professional certifications in Microsoft Azure, Databricks, or modern development frameworks
• Bachelors degree in Computer Science, Data Science, Engineering, or a related technical discipline, or equivalent practical experience
• Desirable: Experience implementing data governance frameworks, data lineage tracking, and role-based access control using tools such as Apache Atlas or Microsoft Purview
• Desirable: Experience building event-driven architectures and real-time data streaming pipelines using tools such as Apache Kafka, AWS Kinesis, or Azure Event Hubs
• Desirable: Familiarity with agentic AI frameworks and architectures such as Microsoft Agent Framework or equivalent frameworks in Python, TypeScript, or other languages
• Desirable: Experience architecting data solutions for scalability, performance optimization, security hardening, and operational excellence in production environments
• Desirable: Knowledge of Infrastructure as Code frameworks including Terraform or Azure Bicep for data platform provisioning and management
• Desirable: Background in professional services, consulting, or client-facing technical delivery environments
• Desirable: Active contribution to open-source data engineering projects or participation in data engineering and cloud communities

Responsibilities:

• Shape the technical direction for data engineering initiatives and services within Technology & Solutions
• Design and deliver enterprise-scale data solutions on cloud platforms, leveraging Databricks as the core data engineering platform
• Build and optimize ETL/ELT data pipelines using PySpark, SQL, and Python, orchestrating complex data workflows through Databricks Workflows, Delta Live Tables, and Azure Data Factory
• Implement and maintain Delta Lake storage architectures and Unity Catalog governance frameworks, ensuring data quality, security, and compliance across the data estate
• Design data solutions for scalability, performance, resilience, and operational excellence, embedding enterprise-grade standards from inception through production deployment
• Lead technical discovery and requirements gathering with senior stakeholders, translating business needs into actionable data platform strategies and technical roadmaps
• Integrate data platforms with business intelligence and analytics tools including Databricks AI/BI and Power BI to enable self-service analytics and data-driven decision making
• Establish, evolve, and enforce data engineering standards, coding practices, and quality gates that enable safe, scalable, and maintainable data platform delivery
• Champion modern software engineering practices within data engineering contexts, including Git version control workflows, automated testing frameworks, and CI/CD deployment pipelines for data workloads
• Drive comprehensive observability, monitoring, and alerting for data pipelines and platforms, ensuring operational readiness and rapid incident response
• Promote AI-augmented data engineering practices, leveraging approved AI coding assistants to enhance productivity while maintaining code quality and standards
• Provide technical direction and engineering accountability for data engineering initiatives, leading teams through hands-on contribution and solving key business challenges
• Coach and mentor data engineers at various career levels, fostering a culture of technical excellence, continuous learning, and innovation adoption
• Manage delivery of data engineering projects using Agile methodologies, balancing technical execution with people leadership and stakeholder management
• Foster collaboration across engineering, architecture, and product teams, breaking down silos and promoting knowledge sharing across the organization
• Represent data engineering in portfolio planning discussions, technical governance forums, and architectural review boards

Technologies:

• Agentic AI
• AI
• AWS
• Azure
• Business Intelligence
• CI/CD
• Cloud
• Databricks
• DevOps
• ETL
• Git
• GitHub
• Support
• Kafka
• Kanban
• Network
• Power BI
• Python
• PySpark
• SQL
• Security
• Terraform
• TypeScript
• Unity
• GameDev
• Architect
• Claude Code
• Copilot
• RBAC

More:

We are KPMG in the UK, part of a global network of firms offering Audit, Legal, Tax and Advisory services. With over 16,000 colleagues, we bring creativity, insight and experience to solve our clients and communities biggest problems. We have been doing this for more than 150 years. We aim to be universally recognised as a place for great people to do their best work, with a collaborative and inclusive culture, technology-enabled delivery, real flexibility, inspiring workspaces, innovative ways to collaborate, and hybrid ways of working. With offices across the UK, we work with everyone from small start-ups and individuals to major multinationals across virtually every industry. Our mission is to support the UK in a connected world, guided by our values: Courage, Integrity, Excellence, Together and For Better. We are an inclusive, equal opportunity employer and are committed to ensuring candidates are treated fairly throughout the recruitment process. We value individuality and diversity of background, experience, and perspective, and believe that helps us deliver the best results for our clients.

last updated 37 week of 2026