[Job - 30760] Master Data Developer, Brazil

Job description

At CI&T, we help large enterprises transform the potential of AI into real business impact with AI Deployment, AI-native execution, and tech-integrated business solutions. With 30 years of experience in technological transformation, we accelerate innovation with expertise in Agentic SDLC, Application modernization, Data & AI, Martech and Business strategy. We are 8,000 CI&Ters across more than 25 countries, collaborating to build solutions with real impact. AI is already part of how we work, evolve, and innovate every day. As CI&T continues to expand its data and analytics capabilities, we are seeking a talented and experienced Data Developer to join our team and drive the evolution of modern data platforms for our clients. This role is critical in designing, building, and optimizing scalable data pipelines and lake architectures that empower data-driven decision-making across the organization. The Data Developer will work with cloud-native solutions to support the entire data lifecycle—from ingestion and transformation to storage optimization and analytics enablement. This position requires strong technical expertise in distributed data processing, deep SQL proficiency, and a solid understanding of cloud infrastructure, particularly within the AWS ecosystem. The ideal candidate will balance performance, cost, and maintainability while contributing to reusable, well-architected data solutions. Responsibilities: Data Pipeline Development & Optimization: Design, build, and maintain robust ETL/ELT processes to ingest, transform, and deliver data across a modern Data Lake architecture Develop and optimize distributed data processing workflows using Python and PySpark to handle large-scale datasets efficiently Implement and refine partitioning strategies for data lake storage frameworks (such as Delta Lake or Apache Iceberg) to balance query performance with storage costs Data Transformation & Modeling: Write, optimize, and translate complex SQL queries involving CTEs, window functions, conditional expressions, and aggregations Migrate and modernize data pipelines from legacy RDBMS platforms to cloud-native analytics environments Leverage object-oriented programming principles to contribute to in-house libraries for code reusability and standardization Cloud Infrastructure & Orchestration: Work confidently with AWS-native services including Glue (Jobs, Catalog, Triggers, Workflows), Athena, Redshift, S3, Lambda, EventBridge, and related data services Collaborate with infrastructure and DevOps teams to provision and manage data resources using Infrastructure as Code (IaC) tools such as CloudFormation, CDK, or Terraform Monitor data pipeline health and performance using CloudWatch and other observability tools, proactively addressing issues and improving reliability Data Governance & Quality: Ensure data integrity, consistency, and compliance across pipelines and storage layers Implement metric tracking and observability frameworks to provide transparency into data workflows and SLAs Support data catalog management and metadata governance practices Collaboration & Continuous Improvement: Partner with data analysts, scientists, and business stakeholders to understand requirements and translate them into scalable technical solutions Contribute to technical documentation, code reviews, and knowledge sharing within the team Stay current with emerging data engineering practices, tools, and cloud-native innovations Requirements: Solid experience working with ETL processes and data pipeline development with AWS Strong proficiency in Python as the primary programming language, with demonstrated experience writing and optimizing PySpark code for distributed data processing Thorough understanding of SQL, including complex queries (CTEs, window functions, aggregations, conditional expressions) and experience translating workloads from legacy RDBMS platforms Hands-on experience with AWS Glue (Jobs, Catalog, Triggers, Workflows), Athena, and Redshift Solid understanding of Data Lake architectures and partitioning strategies to optimize performance and cost Good understanding of object-oriented programming (OOP) principles and experience working with reusable code libraries Comfortable working with Git, Shell scripts, and Linux environments Familiarity with observability, monitoring, and metric tracking practices English Advanced/Fluent Nice to Have: Experience with open table formats such as Delta Lake or Apache Iceberg Knowledge of TypeScript Practical experience with Infrastructure as Code (IaC) tools such as CloudFormation (preferred), CDK, or Terraform Familiarity with additional AWS services such as SageMaker AI, ECS, RDS, DynamoDB, IAM, or EventBridge Experience working with Pandas for data manipulation and analysis Exposure to machine learning workflows or AI-driven data initiatives #LI-JP3 At CI&T, we help large enterprises transform the potential of AI into real business impact with AI Deployment, AI-native execution, and tech-integrated business solutions. With 30 years of experience in technological transformation, we accelerate innovation with expertise in Agentic SDLC, Application modernization, Data & AI, Martech and Business strategy. We are 8,000 CI&Ters across more than 25 countries, collaborating to build solutions with real impact. AI is already part of how we work, evolve, and innovate every day. As CI&T continues to expand its data and analytics capabilities, we are seeking a talented and experienced Data Developer to join our team and drive the evolution of modern data platforms for our clients. This role is critical in designing, building, and optimizing scalable data pipelines and lake architectures that empower data-driven decision-making across the organization. The Data Developer will work with cloud-native solutions to support the entire data lifecycle—from ingestion and transformation to storage optimization and analytics enablement. This position requires strong technical expertise in distributed data processing, deep SQL proficiency, and a solid understanding of cloud infrastructure, particularly within the AWS ecosystem. The ideal candidate will balance performance, cost, and maintainability while contributing to reusable, well-architected data solutions. Responsibilities: Data Pipeline Development & Optimization: Design, build, and maintain robust ETL/ELT processes to ingest, transform, and deliver data across a modern Data Lake architecture Develop and optimize distributed data processing workflows using Python and PySpark to handle large-scale datasets efficiently Implement and refine partitioning strategies for data lake storage frameworks (such as Delta Lake or Apache Iceberg) to balance query performance with storage costs Data Transformation & Modeling: Write, optimize, and translate complex SQL queries involving CTEs, window functions, conditional expressions, and aggregations Migrate and modernize data pipelines from legacy RDBMS platforms to cloud-native analytics environments Leverage object-oriented programming principles to contribute to in-house libraries for code reusability and standardization Cloud Infrastructure & Orchestration: Work confidently with AWS-native services including Glue (Jobs, Catalog, Triggers, Workflows), Athena, Redshift, S3, Lambda, EventBridge, and related data services Collaborate with infrastructure and DevOps teams to provision and manage data resources using Infrastructure as Code (IaC) tools such as CloudFormation, CDK, or Terraform Monitor data pipeline health and performance using CloudWatch and other observability tools, proactively addressing issues and improving reliability Data Governance & Quality: Ensure data integrity, consistency, and compliance across pipelines and storage layers Implement metric tracking and observability frameworks to provide transparency into data workflows and SLAs Support data catalog management and metadata governance practices Collaboration & Continuous Improvement: Partner with data analysts, scientists, and business stakeholders to understand requirements and translate them into scalable technical solutions Contribute to technical documentation, code reviews, and knowledge sharing within the team Stay current with emerging data engineering practices, tools, and cloud-native innovations Requirements: Solid experience working with ETL processes and data pipeline development with AWS Strong proficiency in Python as the primary programming language, with demonstrated experience writing and optimizing PySpark code for distributed data processing Thorough understanding of SQL, including complex queries (CTEs, window functions, aggregations, conditional expressions) and experience translating workloads from legacy RDBMS platforms Hands-on experience with AWS Glue (Jobs, Catalog, Triggers, Workflows), Athena, and Redshift Solid understanding of Data Lake architectures and partitioning strategies to optimize performance and cost Good understanding of object-oriented programming (OOP) principles and experience working with reusable code libraries Comfortable working with Git, Shell scripts, and Linux environments Familiarity with observability, monitoring, and metric tracking practices English Advanced/Fluent Nice to Have: Experience with open table formats such as Delta Lake or Apache Iceberg Knowledge of TypeScript Practical experience with Infrastructure as Code (IaC) tools such as CloudFormation (preferred), CDK, or Terraform Familiarity with additional AWS services such as SageMaker AI, ECS, RDS, DynamoDB, IAM, or EventBridge Experience working with Pandas for data manipulation and analysis Exposure to machine learning workflows or AI-driven data initiatives #LI-JP3 Our benefits: -Health and dental insurance -Meal and food allowance -Childcare assistance -Extended paternity leave -Partnership with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass; -Profit Sharing and Results Participation (PLR); -Life insurance -Continuous learning platform (CI&T University); -Discount club -Free online platform dedicated to physical, mental, and overall well-being -Pregnancy and responsible parenting course -Partnerships with online learning platforms -Language learning platform And many more! More details about our benefits here: https://ciandt.com/br/pt-br/carreiras At CI&T, inclusion starts at the first contact. If you are a person with a disability, it is important to present your assessment during the selection process. See which data needs to be included in the report by clicking here.This way, we can ensure the support and accommodations that you deserve. If you do not yet have the assessment, don't worry: we can support you in obtaining it. We have a dedicated Health and Well-being team, inclusion specialists, and affinity groups who will be with you at every stage. Count on us to make this journey side by side. As CI&T continues to expand its data and analytics capabilities, we are seeking a talented and experienced Data Developer to join our team and drive the evolution of modern data platforms for our clients. This role is critical in designing, building, and optimizing scalable data pipelines and lake architectures that empower data-driven decision-making across the organization. The Data Developer will work with cloud-native solutions to support the entire data lifecycle—from ingestion and transformation to storage optimization and analytics enablement. This position requires strong technical expertise in distributed data processing, deep SQL proficiency, and a solid understanding of cloud infrastructure, particularly within the AWS ecosystem. The ideal candidate will balance performance, cost, and maintainability while contributing to reusable, well-architected data solutions. Responsibilities: Data Pipeline Development & Optimization: Design, build, and maintain robust ETL/ELT processes to ingest, transform, and deliver data across a modern Data Lake architecture Develop and optimize distributed data processing workflows using Python and PySpark to handle large-scale datasets efficiently Implement and refine partitioning strategies for data lake storage frameworks (such as Delta Lake or Apache Iceberg) to balance query performance with storage costs Data Transformation & Modeling: Write, optimize, and translate complex SQL queries involving CTEs, window functions, conditional expressions, and aggregations Migrate and modernize data pipelines from legacy RDBMS platforms to cloud-native analytics environments Leverage object-oriented programming principles to contribute to in-house libraries for code reusability and standardization Cloud Infrastructure & Orchestration: Work confidently with AWS-native services including Glue (Jobs, Catalog, Triggers, Workflows), Athena, Redshift, S3, Lambda, EventBridge, and related data services Collaborate with infrastructure and DevOps teams to provision and manage data resources using Infrastructure as Code (IaC) tools such as CloudFormation, CDK, or Terraform Monitor data pipeline health and performance using CloudWatch and other observability tools, proactively addressing issues and improving reliability Data Governance & Quality: Ensure data integrity, consistency, and compliance across pipelines and storage layers Implement metric tracking and observability frameworks to provide transparency into data workflows and SLAs Support data catalog management and metadata governance practices Collaboration & Continuous Improvement: Partner with data analysts, scientists, and business stakeholders to understand requirements and translate them into scalable technical solutions Contribute to technical documentation, code reviews, and knowledge sharing within the team Stay current with emerging data engineering practices, tools, and cloud-native innovations Requirements: Solid experience working with ETL processes and data pipeline development with AWS Strong proficiency in Python as the primary programming language, with demonstrated experience writing and optimizing PySpark code for distributed data processing Thorough understanding of SQL, including complex queries (CTEs, window functions, aggregations, conditional expressions) and experience translating workloads from legacy RDBMS platforms Hands-on experience with AWS Glue (Jobs, Catalog, Triggers, Workflows), Athena, and Redshift Solid understanding of Data Lake architectures and partitioning strategies to optimize performance and cost Good understanding of object-oriented programming (OOP) principles and experience working with reusable code libraries Comfortable working with Git, Shell scripts, and Linux environments Familiarity with observability, monitoring, and metric tracking practices English Advanced/Fluent Nice to Have: Experience with open table formats such as Delta Lake or Apache Iceberg Knowledge of TypeScript Practical experience with Infrastructure as Code (IaC) tools such as CloudFormation (preferred), CDK, or Terraform Familiarity with additional AWS services such as SageMaker AI, ECS, RDS, DynamoDB, IAM, or EventBridge Experience working with Pandas for data manipulation and analysis Exposure to machine learning workflows or AI-driven data initiatives #LI-JP3 At CI&T, we help large enterprises transform the potential of AI into real business impact with AI Deployment, AI-native execution, and tech-integrated business solutions. With 30 years of experience in technological transformation, we accelerate innovation with expertise in Agentic SDLC, Application modernization, Data & AI, Martech and Business strategy. We are 8,000 CI&Ters across more than 25 countries, collaborating to build solutions with real impact. AI is already part of how we work, evolve, and innovate every day. As CI&T continues to expand its data and analytics capabilities, we are seeking a talented and experienced Data Developer to join our team and drive the evolution of modern data platforms for our clients. This role is critical in designing, building, and optimizing scalable data pipelines and lake architectures that empower data-driven decision-making across the organization. The Data Developer will work with cloud-native solutions to support the entire data lifecycle—from ingestion and transformation to storage optimization and analytics enablement. This position requires strong technical expertise in distributed data processing, deep SQL proficiency, and a solid understanding of cloud infrastructure, particularly within the AWS ecosystem. The ideal candidate will balance performance, cost, and maintainability while contributing to reusable, well-architected data solutions. Responsibilities: Data Pipeline Development & Optimization: Design, build, and maintain robust ETL/ELT processes to ingest, transform, and deliver data across a modern Data Lake architecture Develop and optimize distributed data processing workflows using Python and PySpark to handle large-scale datasets efficiently Implement and refine partitioning strategies for data lake storage frameworks (such as Delta Lake or Apache Iceberg) to balance query performance with storage costs Data Transformation & Modeling: Write, optimize, and translate complex SQL queries involving CTEs, window functions, conditional expressions, and aggregations Migrate and modernize data pipelines from legacy RDBMS platforms to cloud-native analytics environments Leverage object-oriented programming principles to contribute to in-house libraries for code reusability and standardization Cloud Infrastructure & Orchestration: Work confidently with AWS-native services including Glue (Jobs, Catalog, Triggers, Workflows), Athena, Redshift, S3, Lambda, EventBridge, and related data services Collaborate with infrastructure and DevOps teams to provision and manage data resources using Infrastructure as Code (IaC) tools such as CloudFormation, CDK, or Terraform Monitor data pipeline health and performance using CloudWatch and other observability tools, proactively addressing issues and improving reliability Data Governance & Quality: Ensure data integrity, consistency, and compliance across pipelines and storage layers Implement metric tracking and observability frameworks to provide transparency into data workflows and SLAs Support data catalog management and metadata governance practices Collaboration & Continuous Improvement: Partner with data analysts, scientists, and business stakeholders to understand requirements and translate them into scalable technical solutions Contribute to technical documentation, code reviews, and knowledge sharing within the team Stay current with emerging data engineering practices, tools, and cloud-native innovations Requirements: Solid experience working with ETL processes and data pipeline development with AWS Strong proficiency in Python as the primary programming language, with demonstrated experience writing and optimizing PySpark code for distributed data processing Thorough understanding of SQL, including complex queries (CTEs, window functions, aggregations, conditional expressions) and experience translating workloads from legacy RDBMS platforms Hands-on experience with AWS Glue (Jobs, Catalog, Triggers, Workflows), Athena, and Redshift Solid understanding of Data Lake architectures and partitioning strategies to optimize performance and cost Good understanding of object-oriented programming (OOP) principles and experience working with reusable code libraries Comfortable working with Git, Shell scripts, and Linux environments Familiarity with observability, monitoring, and metric tracking practices English Advanced/Fluent Nice to Have: Experience with open table formats such as Delta Lake or Apache Iceberg Knowledge of TypeScript Practical experience with Infrastructure as Code (IaC) tools such as CloudFormation (preferred), CDK, or Terraform Familiarity with additional AWS services such as SageMaker AI, ECS, RDS, DynamoDB, IAM, or EventBridge Experience working with Pandas for data manipulation and analysis Exposure to machine learning workflows or AI-driven data initiatives #LI-JP3