Data Engineer

Job description

**This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineer based in Ireland.** This role offers the opportunity to help build, maintain, and modernize a growing data platform in a cloud-first environment. You will design reliable data pipelines and scalable data solutions that enable teams to access and use high-quality information effectively. The role combines modern AWS infrastructure with opportunities to improve and transition legacy data processes. You will work across batch and real-time data workflows, APIs, integrations, and data warehouse solutions. Collaboration with analytics teams and other stakeholders will be central to understanding requirements and delivering practical solutions. This is a hands-on opportunity to improve data reliability, performance, security, and quality while contributing to the evolution of the broader data architecture. ### Accountabilities - Design, develop, and maintain scalable and reliable data pipelines and Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes. - Develop, maintain, and optimize data warehouse solutions and data models. - Build and maintain data infrastructure primarily on Amazon Web Services (AWS), while supporting relevant legacy systems. - Develop APIs and data integrations to enable effective data exchange across systems. - Contribute to real-time data pipelines and processing using Apache Kafka. - Automate and optimize data processes, workflows, and operational tasks. - Monitor, troubleshoot, and continuously improve the reliability, performance, security, and data quality of pipelines and infrastructure. - Work closely with analytics teams and other stakeholders to understand data requirements and deliver effective technical solutions. - Maintain accurate and up-to-date documentation covering pipelines, infrastructure, workflows, and operational processes. - Contribute to the modernization of legacy data processes, infrastructure, and overall data architecture. ### **Requirements** - 2+ years of professional experience in data engineering. - Strong SQL and Python programming skills. - Hands-on experience with AWS services, particularly Amazon S3, Amazon Redshift, AWS Glue, and AWS Lambda. - Practical experience with Apache Airflow for data workflow orchestration. - Strong understanding of data warehousing principles and data modeling. - Experience designing, developing, and maintaining production-grade data pipelines, with a strong focus on reliability, monitoring, and data quality. - Solid understanding of both relational and non-relational databases. - Familiarity with Git and Continuous Integration/Continuous Deployment (CI/CD) practices, preferably using GitLab. - Experience developing APIs and building data integrations. - Strong written and verbal communication skills. - Upper-intermediate English proficiency or higher. - Experience with Kafka and real-time data processing is a plus. - Experience with Microsoft SQL Server and SQL Server Integration Services (SSIS) is a plus. ### **Benefits** - Full-time position. - Opportunity to work within an Analytics-focused environment. - Remote working arrangement. - Opportunity to contribute to modern cloud-based data infrastructure. - Hands-on exposure to AWS, Airflow, Kafka, data warehousing, and data modernization initiatives. - Opportunity to contribute to the improvement of data reliability, quality, performance, and architecture. - Collaborative work with analytics teams and cross-functional stakeholders.