Senior Data Engineer - Corsham

Salary: £129,792 - 129,792 per year

Requirements:

• Proven experience delivering data engineering solutions in a project or production environment.
• Strong Python and SQL skills.
• Hands-on experience with Apache NiFi and data pipeline orchestration.
• Experience with data modelling and schema design.
• Experience with Parquet and S3-compatible object storage.
• Experience with REST API integration.
• Experience with OpenShift or Kubernetes.
• Experience with GitLab CI/CD practices and pipelines.
• Experience developing data contracts or Interface Control Documents (ICDs).
• Experience with data validation and assurance.
• Experience with metadata management, lineage and traceability.
• Critical thinking, problem-solving and effective stakeholder communication.
• Degree in STEM, Data, AI, or a programming-related field.
• Experience influencing technical decisions and supporting capability growth.
• Experience with big data tools such as Spark, Hadoop, or NiFi.
• Exposure to Ansible and Terraform.
• Understanding of ML algorithms and deployment.

Responsibilities:

• Design, build and maintain secure, scalable data solutions across the full lifecycle, including acquisition, integration, storage, processing and analysis.
• Develop and maintain data ingestion and transformation pipelines using Apache NiFi, Python and SQL.
• Design data models, schemas and data contracts or Interface Control Documents (ICDs) to support reliable integration between systems.
• Integrate data from REST APIs and other source systems, working with Parquet and S3-compatible object storage.
• Implement data validation, assurance, metadata management and traceability to ensure data is accurate, complete and fit for purpose.
• Develop tools for data-driven insights and task automation, and research and adopt appropriate emerging technologies.
• Collaborate with customer and internal stakeholders to define data, interface and tool requirements.
• Work closely with DevOps Engineers, Data Scientists, Analysts and Cyber Consultants to deliver integrated solutions.
• Take ownership of deliverables and ensure the quality, security and maintainability of outputs.

Technologies:

• AI
• API
• Ansible
• Big Data
• CI/CD
• DevOps
• GitLab
• Hadoop
• Support
• Kubernetes
• OpenShift
• Python
• REST
• SQL
• Security
• Spark
• Terraform

More:

We are seeking a Senior Data Engineer to work full time at client sites in Wiltshire, with potential travel to other UK sites. This role is inside IR35 and may offer the opportunity to condense the hours of work. Active DV clearance is required for the role.

last updated 39 week of 2026