About the company
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Software Data Engineer (AWS) based in United Kingdom.
Responsibilities
- Design and develop scalable streaming data pipelines using Kafka/MSK, Flink or Spark Streaming.
- Build batch transformation pipelines within a medallion lakehouse architecture on Amazon S3 using Iceberg and Parquet.
- Design and manage data storage solutions across Aurora/RDS PostgreSQL, DynamoDB, and object-based lakehouse storage.
- Design, deploy, and maintain AWS infrastructure using EKS, Fargate, Lambda, S3, VPC, Aurora/RDS, and DynamoDB.
- Take data solutions from prototype through to production by refactoring, optimizing, and applying CI/CD, containerization, and deployment best practices.
- Collaborate with international teams to gather requirements and define data loading and ingestion processes.
- Document technical developments, perform integration testing, and manage deployments within AWS environments.
- Monitor production pipelines and ensure observability, data quality, performance, and reliability.
Requirements
- Master's degree in Computer Science, Computer Engineering, or a related technical discipline; a PhD is a plus.
- Strong programming and data engineering skills, particularly with Python and SQL.
- Hands-on experience building streaming data pipelines using Kafka/MSK and Flink or Spark Streaming.
- Practical experience with AWS services including EKS, Fargate, Lambda, S3, Aurora/RDS PostgreSQL, DynamoDB, and VPC.
- Solid understanding of big data technologies such as Spark, Hive, and HDFS, together with modern lakehouse formats such as Iceberg and Parquet.
- Experience with Git, CI/CD, Docker, Terraform or other Infrastructure as Code tools, REST APIs, and Agile/Scrum methodologies.
- Understanding of scalable data architectures, production data pipelines, data quality, and monitoring.
- Experience with Iceberg or Delta Lake, change data capture (CDC), schema evolution at scale, or Scala is an advantage.
- Strong communication skills and the ability to collaborate effectively with cross-functional and international teams.
Conditions
- Competitive compensation package aligned with experience, skills, and qualifications.
- Gross annual base salary starting from €29,500, with the possibility of a higher offer based on the candidate's profile.
- Employment contract governed by the applicable Gomma Plastica National Collective Bargaining Agreement.
- Welfare and employee benefits package.
- Restaurant meal/ticket benefits.
- Flexible working hours.
- Hybrid remote working model.
- Opportunities for professional development and career progression.