Big Data DevOps Engineer

Salary: £43,000 - 83,000 per year

Requirements:

• Strong DevOps expertise with CI/CD pipelines in a Big Data technology landscape.
• Hands-on experience with AWS, Hortonworks, S3, Java 8, HBase, Kafka, JanusGraph, HDF (NiFi), and related big data technologies.
• Strong hands-on skills in setting up Linux-based infrastructure.
• Experience with infrastructure automation tools such as Ansible, Terraform, SaltStack, Puppet, and Docker.
• Experience with AWS cloud setup and administration.
• Fluency in Python and Java.
• Expertise in CI tools such as Jenkins and Nexus.
• Experience with Java build systems, primarily Maven and Gradle.
• Experience supporting code versioning and branching strategies using GitHub.
• Experience with configuration management and backup/restore policies using specialized tools and custom Bash scripts.
• Experience with system monitoring solutions such as Zabbix, Prometheus, and Nagios.
• Experience with log management using Elasticsearch and Logstash (ELK).
• Experience with Hadoop cluster administration, maintenance, capacity planning, and optimization.
• Experience securing Hadoop infrastructure using Knox and Kerberos.
• Experience working in an Agile environment.
• Ability to clearly articulate the technology and business pros and cons of various technologies, platforms, and architectural options.
• Ability to document processes.
• Excellent communication skills, with a can-do attitude and a team-player mindset.
• Excellent written and verbal communication skills and the ability to work in complex and critical systems.
• Graduate with work experience.
• Hortonworks Certified Administrator (HDPCA).
• Eligibility for SC clearance: UK/EU citizen or indefinite leave to remain visa holder without leaving the UK for more than 6 months in the last 5 years.

Responsibilities:

• Represent us as a DevOps expert and demonstrate our value to clients, prospects, and partners to establish trust, win confidence, and drive more business.
• Design, deploy, and maintain complex Linux-based infrastructure.
• Set up highly secure Hortonworks Hadoop clusters in AWS cloud.
• Perform cluster administration, maintenance, upgrades, capacity planning, and optimization.
• Automate operations by setting up continuous integration and continuous deployment pipelines, code versioning, and application containerisation.
• Run and manage a data platform with multiple tenants and client applications for transactional and analytical use cases.
• Oversee the integration of development output into production-quality releases and support the live system.
• Work with other leads and developers, own and contribute to deliverables, and guide the team towards success.
• Identify areas where processes and standards can be improved and help implement changes.

Technologies:

• AWS
• Ansible
• Big Data
• Bash
• CI/CD
• Cloud
• DevOps
• Docker
• ELK
• ElasticSearch
• GitHub
• Gradle
• Hadoop
• Support
• Java
• Jenkins
• Kafka
• Linux
• Maven
• Nexus
• Prometheus
• Puppet
• Python
• Terraform
• Zabbix

More:

We are hiring for a new transformational program for a UK customer focused on big data and advanced visualization capabilities. We work in an Agile environment with teams co-located in London, and this is a consultancy role. Successful applicants will work at our clients, primarily based in and around London, with some exceptions. The role is permanent and offers a competitive salary.

last updated 36 week of 2026