Software Data Engineer (AWS)

Job description

**This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Software Data Engineer (AWS) based in Spain.** As a Software Data Engineer, you will contribute to the development of a modern data platform supporting manufacturing operations across a global industrial environment. You will design and build scalable streaming and batch data pipelines that connect plant systems and enable reliable, data-driven decision-making. The role combines cloud engineering, big data, data integration, and software development, with a strong focus on AWS technologies. You will work with Kafka, Flink or Spark Streaming, lakehouse architectures, and Infrastructure as Code to transform complex data into production-ready solutions. You will take ownership of solutions from initial design and prototyping through deployment, optimization, monitoring, and ongoing maintenance. Working closely with international teams, you will help modernize legacy data solutions and contribute to broader digital transformation initiatives. This is an opportunity to work with advanced data technologies while having a direct impact on the efficiency, reliability, and scalability of critical manufacturing data systems. ### Accountabilities - Design and develop scalable streaming data pipelines using Kafka/MSK, Flink or Spark Streaming, and reliable routing into curated data storage. - Build batch transformation pipelines within a medallion lakehouse architecture, supporting bronze, silver, and gold data layers on Amazon S3 and using formats such as Iceberg and Parquet. - Design and manage data storage solutions across relational databases such as Aurora/RDS PostgreSQL, NoSQL technologies including DynamoDB, and object-based lakehouse storage. - Design, deploy, and maintain AWS infrastructure supporting data and machine learning workloads, using services such as EKS, Fargate, Lambda, S3, VPC, Aurora/RDS, and DynamoDB. - Take data solutions from prototype through to production by refactoring, optimizing, and applying software engineering, CI/CD, containerization, and deployment best practices. - Collaborate with international teams to gather requirements, contribute to solution design, and define effective data loading and ingestion processes. - Document technical developments, perform integration testing, and manage deployments within AWS environments. - Support the transition of completed solutions to the relevant support and governance teams, ensuring reliable operation and clear ownership. - Monitor production pipelines and ensure strong observability, data quality, performance, reliability, and maintainability. - Participate in an application maintenance and support environment, investigating user-reported issues and implementing improvements to data ingestion processes. ### Requirements: - Master's degree in Computer Science, Computer Engineering, or a related technical discipline; a PhD is considered a plus. - Strong programming and data engineering skills, particularly with **Python and SQL**. - Hands-on experience building streaming data pipelines using **Kafka/MSK** and **Flink** or **Spark Streaming**. - Practical experience with AWS services including **EKS, Fargate, Lambda, S3, Aurora/RDS PostgreSQL, DynamoDB, and VPC**. - Solid understanding of big data technologies such as **Spark, Hive, and HDFS**, together with modern lakehouse formats such as **Iceberg and Parquet**. - Experience with **Git, CI/CD, Docker, Terraform or other Infrastructure as Code tools, REST APIs**, and Agile/Scrum methodologies. - Understanding of scalable data architectures, production data pipelines, data quality, monitoring, and operational best practices. - Experience with Iceberg or Delta Lake, change data capture (CDC), schema evolution at scale, or Scala is considered an advantage. - Strong communication skills and the ability to collaborate effectively with cross-functional and international teams. - Curiosity and enthusiasm for learning new technologies, products, and technical capabilities. - Strong attention to detail, organization, prioritization, and time-management skills. - Ability to work independently while maintaining effective collaboration with technical and business stakeholders. ### Benefits: - Competitive compensation package aligned with experience, skills, and qualifications. - Gross annual base salary starting from **€29,500**, with the possibility of a higher offer based on the candidate's profile. - Employment contract governed by the applicable **Gomma Plastica National Collective Bargaining Agreement**. - Welfare and employee benefits package, with further details provided during the recruitment process and subject to applicable policies. - Restaurant meal/ticket benefits. - Flexible working hours. - Hybrid remote working model. - Opportunities for professional development and career progression. - Opportunity to work with advanced cloud, big data, AI, and digital transformation technologies in an international environment. - Inclusive workplace committed to equal opportunity and non-discrimination.