Engenheiro de Dados Sênior

Job description

**This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Senior Data Engineer (Engenheiro de Dados Sênior) based in Brazil.** Join a technology-driven environment where data architecture and engineering play a central role in enabling advanced analytics, streaming, and artificial intelligence initiatives. As a Senior Data Engineer, you will design and develop scalable, secure, and high-performance data platforms using Microsoft Azure and Databricks. You will lead the implementation of Lakehouse architectures, build robust data pipelines, and ensure information remains reliable, accessible, and well-governed. Working closely with technical stakeholders and international teams, you will help transform complex data requirements into efficient, production-ready solutions. Your expertise will contribute to improving data quality, optimizing platform performance, and enabling data-driven decision-making across the organization. This fully remote, long-term opportunity is ideal for a senior professional who combines deep technical expertise with strong communication skills and the ability to collaborate confidently with clients and teams across North America. ### Accountabilities: - **Data Architecture and Solution Design:** Design, develop, and maintain scalable data architectures using Microsoft Azure and Databricks, ensuring alignment with business objectives, technical requirements, and long-term platform needs. - **Lakehouse Architecture Implementation:** Build and optimize modern Lakehouse solutions using Delta Lake and Unity Catalog to support reliable data storage, processing, access management, and analytics. - **Data Pipeline Development:** Design, implement, maintain, and optimize robust data pipelines that support efficient ingestion, transformation, integration, and delivery of data across multiple sources. - **Advanced Data Processing:** Leverage Apache Spark and PySpark to develop high-performance data processing solutions capable of handling large volumes of structured and unstructured information. - **Streaming and Real-Time Data:** Support the development and optimization of streaming data architectures and processing workflows to enable timely insights and data-driven applications. - **Data Quality and Reliability:** Establish and implement data quality controls, validation processes, and monitoring mechanisms to ensure accuracy, consistency, completeness, and reliability across data platforms. - **Data Governance and Security:** Implement and maintain data governance frameworks, access controls, security policies, and metadata management practices to protect sensitive information and support compliance requirements. - **Platform Performance and Optimization:** Monitor and optimize data workloads, infrastructure utilization, processing performance, and platform costs to ensure scalable and efficient operations. - **Analytics and AI Enablement:** Develop and maintain data foundations that support business intelligence, advanced analytics, machine learning, and artificial intelligence initiatives. - **Technical Collaboration:** Work closely with engineering teams, analytics specialists, business stakeholders, and international clients to translate requirements into effective technical solutions. - **Technical Documentation and Best Practices:** Maintain architecture documentation, establish engineering standards, and promote best practices in data modeling, pipeline development, governance, and platform management. - **International Stakeholder Communication:** Communicate technical concepts, architectural decisions, project progress, and recommendations clearly in English when collaborating with clients and teams in the United States and Canada. ## Requirements: - **Professional Experience:** Extensive experience as a Senior Data Engineer and/or Data Architect, with a proven track record of designing and implementing enterprise-scale data solutions. - **Databricks and Microsoft Azure:** Advanced knowledge of Databricks and Microsoft Azure, including experience designing, deploying, and managing cloud-based data engineering platforms. - **Lakehouse Architecture:** Strong experience with Lakehouse architecture principles, Delta Lake, and Unity Catalog, including data management, governance, and access control. - **Apache Spark and PySpark:** Advanced proficiency in Apache Spark and PySpark for large-scale data processing, transformation, and workload optimization. - **Data Pipeline Engineering:** Demonstrated experience building, maintaining, and optimizing scalable data pipelines and modern data platforms, with an understanding of performance, reliability, and operational efficiency. - **Data Engineering and Analytics:** Strong understanding of data engineering principles, analytics requirements, data integration, and the preparation of reliable datasets for downstream consumption. - **Streaming Technologies:** Experience designing or supporting streaming data solutions and processing architectures for real-time or near-real-time applications. - **Data Governance and Security:** Knowledge of data quality management, metadata, governance frameworks, access controls, security practices, and data lifecycle management. - **Performance Optimization:** Ability to identify bottlenecks, troubleshoot complex data workloads, and improve processing efficiency, scalability, and resource utilization. - **Advanced English Proficiency:** Fluent English at an expert professional level, with the ability to communicate technical information effectively and collaborate directly with clients and teams in the United States and Canada. - **Communication and Collaboration:** Strong interpersonal and communication skills, with the ability to work effectively across multidisciplinary and geographically distributed teams. - **Proactivity and Ownership:** A proactive approach to problem-solving, with the ability to take initiative, anticipate challenges, and drive technical tasks through to completion. - **Prioritization and Adaptability:** Strong organizational skills, a sense of urgency, and the ability to prioritize competing demands while maintaining high standards of quality. - **Remote Work Readiness:** Ability to work independently in a fully remote environment while maintaining effective communication, accountability, and collaboration with international stakeholders. - **Contract Availability:** Willingness to work under a Brazilian service-provider arrangement (Pessoa Jurídica – PJ) for an indefinite-term engagement. ## Benefits: - **Fully Remote Work:** Work remotely from Brazil, with flexibility in managing your work environment. - **Long-Term Opportunity:** Indefinite-term contract offering the potential for continued professional engagement. - **Learning and Development:** Access to learning incentive programs, including Udemy resources, to support technical development and continuous learning. - **Corporate English Classes:** Access to corporate English lessons at affordable rates to strengthen professional communication skills. - **International Collaboration:** Work with clients and technical teams in the United States and Canada, gaining exposure to international projects and engineering practices. - **Technical Growth:** Expand your expertise in Databricks, Microsoft Azure, Lakehouse architecture, advanced data processing, and modern data platforms. - **Meaningful Technical Impact:** Contribute to scalable data solutions that support analytics, streaming, and AI initiatives across business environments. - **Collaborative Environment:** Work with multidisciplinary professionals in an environment that values innovation, knowledge sharing, proactive problem-solving, and technical excellence.