Senior AWS Data Engineer

Salary: £63,000 - 103,000 per year

Requirements:

• Strong experience developing telecom network performance data parsers for multiple file formats, including XML and ASN.1, and converting them into Parquet columnar output
• Knowledge or experience with Iceberg table format preferred
• Thorough knowledge of building audit logging and metadata in DynamoDB and forwarding details via CloudWatch
• Strong understanding of 15/60 PM counter granularity and aggregation logic
• Experience building and optimizing distributed data processing jobs using PySpark on AWS EMR
• Experience deploying jobs on data warehouse platforms such as Amazon Redshift and ClickHouse
• Experience working with Amazon MSK (Kafka) for real-time data ingestion and streaming
• Experience automating pipeline orchestration with AWS Step Functions
• Ability to ensure schemas are aligned with vendor (OEM) specifications and business requirements

Responsibilities:

• Develop and write telecom network performance data parsers for multiple file formats and convert them into Parquet columnar output
• Implement AWS Lambda functions triggered by S3 event notifications via SQS queues for high-throughput parsing
• Build robust error handling with retries, DLQ, and SNS notifications
• Build schema management aligned with vendor mapping files and write code to dynamically update schemas with new counters, including automation where possible
• Support schema deployment, upgrades, and rollback processes
• Handle schema upserts and merges to insert new records and update existing ones
• Own the end-to-end lifecycle of schemas, including versioning, updates, and retirement
• Write aggregation code for 15/60 PM counter granularity
• Build and optimize distributed data processing jobs using PySpark on AWS EMR and deploy them to data warehouse platforms
• Work with Amazon MSK (Kafka) for real-time data ingestion and streaming
• Automate pipeline orchestration with AWS Step Functions
• Ensure schemas remain aligned with vendor (OEM) specifications and business requirements

Technologies:

• AI
• AWS
• Lambda
• Redshift
• ClickHouse
• Cloud
• CloudWatch
• Data Warehouse
• Support
• Kafka
• Network
• PySpark
• XML

More:

Tech Mahindra provides technology consulting and digital solutions to global enterprises across industries, helping enable transformative scale at speed. We have 149k+ professionals across 90+ countries supporting 1100+ clients with services spanning consulting, information technology, enterprise applications, business process services, engineering services, network services, customer experience and design, AI and analytics, and cloud and infrastructure services. We are part of the Mahindra Group, founded in 1945, and we are proud to be recognized for our sustainability leadership. We are an equal opportunity employer and support a diverse workforce at all levels of our company.

last updated 33 week of 2026