AirFlow Developer | Python | Data Engineering

Job description

**This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for an AirFlow Developer | Python | Data Engineering based in Brazil.** We are looking for a skilled Airflow Developer with strong Python expertise to build, maintain, and optimize reliable data pipelines in a collaborative, innovation-driven environment. In this role, you will design and manage Apache Airflow DAGs, develop data processing solutions, and implement efficient ETL/ELT workflows that support business and analytics needs. You will integrate diverse data sources, including APIs, databases, and external services, while ensuring data flows remain accurate, scalable, and dependable. You will also monitor pipeline performance, troubleshoot incidents, and apply observability and error-handling best practices to improve operational reliability. Working alongside data engineering, analytics, and technology professionals, you will contribute to technical discussions and continuously improve data infrastructure. This is an opportunity to make a meaningful impact on data-driven solutions while developing your expertise in modern data engineering practices. ### Accountabilities - Develop, maintain, and enhance data pipelines using Apache Airflow, ensuring workflows are reliable, efficient, and maintainable. - Design, implement, and manage Airflow DAGs with a focus on clear organization, performance, and operational stability. - Build Python-based solutions for data automation, processing, transformation, and integration. - Create, maintain, and optimize ETL/ELT processes to support data availability and quality. - Integrate multiple data sources, APIs, databases, and services into cohesive data workflows. - Monitor pipeline execution, investigate failures, troubleshoot incidents, and resolve performance bottlenecks. - Implement robust error-handling practices, including logging, retries, alerts, and observability mechanisms. - Analyze pipeline performance and identify opportunities to improve processing efficiency, scalability, and reliability. - Apply software development best practices, including version control and collaborative development workflows. - Collaborate with Data Engineering, Analytics, and other technology teams to deliver effective data solutions. - Participate in technical discussions, recommend improvements, and contribute to the ongoing evolution of data engineering practices. ## Requirements - Professional experience working with **Apache Airflow**, including developing, maintaining, and troubleshooting DAGs. - Strong proficiency in **Python**, particularly for automation, data processing, and integration tasks. - Knowledge of SQL and relational databases. - Hands-on experience building and maintaining data pipelines and ETL/ELT workflows. - Experience integrating APIs and working with multiple data sources. - Familiarity with software development best practices, code versioning, and Git. - Strong troubleshooting and problem-solving skills, with the ability to investigate issues and resolve incidents effectively. - Good communication skills and the ability to collaborate with cross-functional teams. - Technical English proficiency sufficient to read and understand technical documentation. **Nice-to-have qualifications:** - Experience with Docker and Kubernetes. - Familiarity with cloud environments and cloud-based data services. - Knowledge of CI/CD tools and practices. - Experience with monitoring and observability tools. - Familiarity with distributed processing and data platforms. - Understanding of data engineering architecture and best practices. - Relevant certifications or coursework in cloud computing, data engineering, or software development. ## Benefits - **Profit-sharing program:** Participation in a profit-sharing and results-based compensation program. - **Healthcare coverage:** Health and dental insurance. - **Flexible benefits:** Access to a flexible benefits card. - **Fitness and wellness:** Access to TotalPass and Zenklub services. - **Language development:** Participation in a language-learning program. - **Professional education:** Access to a partnership with FIAP for learning and development opportunities. - **Flexible work arrangements:** The position is advertised as remote, while the benefits description also references a hybrid work model. The specific arrangement and eligibility for remote work from Brazil should be confirmed during the hiring process. - **Flexible working hours:** Greater flexibility in organizing your work schedule. - **Career development:** Internal training programs and a structured career progression plan. - **Mentorship:** Access to a mentoring program to support professional growth. - **Extended parental leave:** Additional support for employees welcoming a child. - **Childcare assistance:** Access to childcare support benefits. - **Knowledge-sharing initiatives:** Opportunities to participate in live coding sessions, coding dojos, discussion groups, and collaborative learning activities. - **Team well-being:** Access to sports and other initiatives designed to encourage connection and a healthy work-life balance. - **Salary details:** The specific salary range is not provided and should be confirmed during the recruitment process.