Senior Data Engineer - AWS

Salary: £65,000 - 105,000 per year

Requirements:

• Strong proficiency in Python, Scala, or Java.
• Hands-on experience with Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks.
• Practical experience building real-time event streaming pipelines with technologies such as Kafka, Spark Streaming, or Amazon Kinesis in AWS environments.
• Proven expertise in designing modern Data Lakehouse and Data Warehousing architectures.
• Solid understanding of CI/CD, DevOps practices, and GDPR-compliant data governance.
• Experience delivering scalable cloud-based data solutions.
• Experience using AI-enabled tools responsibly to improve engineering efficiency while maintaining human oversight and quality assurance.
• Strong communication and collaboration skills.
• Ability to work effectively across multidisciplinary teams and client environments.
• Bonus: experience with data modelling, schema design, and structured and semi-structured datasets.
• Bonus: experience with Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation.
• Bonus: knowledge of automating cloud-based ingestion and transformation frameworks.
• Bonus: experience delivering data engineering solutions within highly regulated industries such as financial services.
• Bonus: passion for mentoring others, promoting engineering best practices, and exploring responsible AI and automation.

Responsibilities:

• Design and build end-to-end data pipelines leveraging AWS-native services and modern data architectures.
• Collaborate with clients to gather requirements, define technical solutions, and deliver production-grade systems that meet business needs.
• Apply AWS Well-Architected Principles, CI/CD practices, and AI-enabled engineering approaches to deliver scalable, secure, resilient, and high-quality solutions.
• Develop robust, tested, and fault-tolerant data engineering solutions while promoting engineering best practices and data governance.
• Mentor junior engineers and contribute to knowledge sharing.
• Encourage the responsible use of approved AI tools and automation to improve collaboration and delivery.

Technologies:

• AI
• AWS
• Big Data
• CI/CD
• Cloud
• Databricks
• DevOps
• Hadoop
• Hive
• Support
• Java
• Kafka
• Python
• Scala
• Spark

More:

We are Capco, a collaborative, flat, and entrepreneurial consulting firm delivering high-impact technology solutions for Tier 1 financial institutions and helping shape the future of digital transformation across financial services and energy. This is a permanent Senior Data Engineer role based in London with a hybrid working model. We offer continuous learning, training, and industry certifications, along with a people-first benefits package that includes bonus, pension, health and life insurance, mental health support, family-friendly leave, backup care, flexible holiday options, a minimum of 40 hours of annual training, a business coach from day one, and additional perks such as online GP access, Wellhub, travel insurance, Tastecard, season ticket loans, Cycle to Work, and dental insurance. We are committed to accessible recruitment and value individuality through our #BeYourselfAtWork culture.

last updated 34 week of 2026