Principal Data Engineer - London

Salary: £52,000 - 92,000 per year

Requirements:

• Extensive data engineering experience with a strong track record of hands-on execution and delivery in complex data environments.
• Deep practical understanding of database ecosystems used in AI and machine learning infrastructure, including vector databases, NoSQL, and document stores.
• Hands-on experience building, scaling, and shipping large-scale data platforms in production.
• Deep practical experience with distributed data processing frameworks such as Apache Spark, Flink, and Hadoop.
• Strong expertise in message brokers and event streaming platforms such as Apache Kafka and Kinesis.
• End-to-end experience across the data pipeline lifecycle, including workflow orchestration tools such as Apache Airflow and Dagster.
• Hands-on expertise with cloud data warehouses such as Snowflake, BigQuery, and Redshift, and data lake architectures such as Databricks, Delta Lake, and Apache Iceberg.
• Advanced SQL skills and proficiency in Python.
• Strong background in modern software development practices, including testing, code review, CI/CD, and Infrastructure as Code.
• Desirable: extensive, progressive experience leading technical projects and mentoring engineering teams.
• Desirable: hands-on experience with cloud-native infrastructure such as AWS, GCP, or Azure.
• Desirable: experience implementing data observability, monitoring, and alerting frameworks at scale.
• Desirable: familiarity with Anaplan or similar enterprise planning platforms.

Responsibilities:

• Lead the data architecture, design, and deployment of scalable, high-throughput Big Data systems into production environments.
• Architect, deploy, and manage the foundational data systems that support modern AI infrastructure, including vector, NoSQL, and document databases.
• Develop end-to-end data engineering solutions, including robust ETL/ELT pipelines, API services, and data ingestion frameworks.
• Design and build the storage and processing layers powering analytics workloads, including data lakes, data warehouses, distributed file systems, and real-time streaming architectures.
• Engineer feature-rich context pipelines that process large-scale enterprise data, balancing batch and streaming patterns seamlessly.
• Optimise and scale large distributed queries and data transformations to ensure high performance and low latency for end users.
• Implement data quality frameworks to measure and ensure data integrity, reliability, and governance across all data assets.
• Collaborate with analytics, product, and platform teams to build data models that capture the semantics of customer metrics, hierarchies, and relationships.
• Stay current with the modern data stack and big data landscape, evaluating new tools, distributed computing frameworks, and database technologies for potential adoption.

Technologies:

• AI
• Airflow
• API
• AWS
• Redshift
• Architect
• Azure
• Big Data
• BigQuery
• CI/CD
• Cloud
• Databricks
• ETL
• Flink
• GCP
• Hadoop
• Support
• Kafka
• Machine Learning
• NoSQL
• Python
• SQL
• Snowflake
• Spark

More:

We are Anaplan, a team of innovators focused on optimizing business decision-making through our AI-infused scenario planning and analysis platform so our customers can outpace their competition and the market. Our customers include more than 2,400 global companies, including Coca-Cola, LinkedIn, Adobe, LVMH, and Bayer. We foster a Winning Culture built on diversity of thought, ambitious goals, and celebrating wins big and small. We are committed to being strategy-led, values-based, and disciplined in execution, and we offer an environment where people are inspired, connected, developed, and rewarded. We welcome every unique background and are committed to an inclusive workplace where all people are respected and valued.

last updated 34 week of 2026