Databricks Lead Engineer

Salary: £55,000 - 95,000 per year

Requirements:

• Proven experience in data engineering and pipeline development on Databricks and cloud-native platforms.
• Strong consulting values and ability to collaborate effectively in client-facing environments.
• Hands-on expertise across the data lifecycle, including ingestion, transformation, modelling, governance, and consumption.
• Strong problem-solving, analytical, and communication skills.
• Experience leading or mentoring engineering teams to deliver high-quality, scalable data solutions.
• Deep expertise with the Databricks platform, including Spark, PySpark or Scala, Delta Lake, Unity Catalog, and MLflow.
• Proficiency with ETL/ELT tools such as DBT, Matillion, Talend, or equivalent.
• Strong SQL and Python or equivalent-language skills for data manipulation and automation.
• Hands-on experience with cloud platforms such as AWS, Azure, or GCP.
• Familiarity with Databricks Workflows and other orchestration tools.
• Knowledge of data modelling methodologies, including star schemas, Data Vault, Kimball, and Inmon.
• Familiarity with medallion architectures, data lakehouse principles, and distributed data processing.
• Experience with version control tools such as GitHub or Bitbucket and CI/CD pipelines.
• Understanding of data governance, security, and compliance frameworks.
• Exposure to AI/ML workloads is desirable.
• 5–8 years of experience in data engineering, data warehousing, or data architecture roles, including at least 3 years working with Databricks.
• University degree required; a BSc/MSc in Computer Science, Data Engineering, or a related field is preferred.
• Databricks certifications, particularly Data Engineer Professional, are highly desirable.

Responsibilities:

• Engage with clients and deliver data engineering solutions.
• Develop batch and streaming data pipelines.
• Design and implement Databricks and lakehouse architectures.
• Perform data modelling and optimisation using Delta Lake and medallion architecture.
• Collaborate with solution architects, data engineers, developers, ML engineers, analysts, and client stakeholders, including data engineering and analytics leaders.
• Apply data quality, governance, and security practices.
• Deliver high-performing, scalable, and secure data pipelines aligned with client requirements.
• Support client satisfaction and successful adoption of Databricks-based solutions.
• Innovate and improve data engineering practices.
• Contribute to practice growth through reusable assets, accelerators, and technical leadership.

Technologies:

• AI
• AWS
• Azure
• BitBucket
• CI/CD
• Cloud
• Data Vault
• Databricks
• ETL
• GCP
• GitHub
• Support
• MLflow
• Network
• Python
• PySpark
• SQL
• Scala
• Security
• Spark
• Talend
• Unity
• dbt
• GameDev
• Architect

More:

We are NTT DATA, a business and technology services leader serving 75% of the Fortune Global 100. We work to accelerate client success and positively impact society through responsible innovation, with capabilities in enterprise-scale AI, cloud, security, connectivity, data centers, and application services. Our consulting and industry solutions help organizations move confidently and sustainably into the digital future. As a Global Top Employer, we have experts in more than 70 countries and offer access to innovation centers and a network of established and start-up partners. NTT DATA is part of NTT Group, which invests more than $3 billion annually in research and development. We embrace diversity and inclusion and are committed to an environment free of unfair discrimination and harassment. This is a full-time role.

last updated 39 week of 2026