Engenheiro de Dados PL/SR - Databricks

Job description

**This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Engenheiro de Dados PL/SR - Databricks based in Brazil.** This role is focused on building scalable, reliable, and high-impact data solutions using modern cloud and big data technologies. You will work across the full data lifecycle, from data ingestion and transformation to analytics, machine learning, deployment, and production monitoring. The position combines hands-on engineering with architecture, data quality, governance, and business-oriented problem solving. You will work extensively with Databricks, Spark, BigQuery, and Google Cloud Platform to process and operationalize large-scale datasets. The environment is highly collaborative and innovation-driven, with opportunities to contribute to AI, Machine Learning, and Generative AI initiatives. You will also act as a technical reference, helping multidisciplinary teams adopt strong engineering and data practices. ### Accountabilities: - Translate business challenges and opportunities into scalable data, Analytics, AI, and Machine Learning solutions, identifying high-value use cases and connecting technical initiatives to measurable financial, operational, and strategic outcomes. - Design, develop, optimize, and operationalize end-to-end data and Machine Learning solutions, covering data exploration, preparation, transformation, modeling, deployment, and production monitoring. - Build and maintain scalable data processing solutions using technologies such as Databricks, Apache Spark, BigQuery, and Google Cloud Platform, ensuring performance, reliability, availability, and efficient use of cloud resources. - Implement data and MLOps practices that support automation, reproducibility, versioning, pipeline management, model monitoring, performance tracking, and detection of issues such as model drift. - Design and evolve cloud-based data architectures that support large-scale processing, storage, analytics, and consumption while maintaining strong standards for security, governance, quality, and observability. - Establish and promote engineering best practices, including documentation, data governance, monitoring, lifecycle management, and quality controls across data and AI initiatives. - Act as a technical reference for Data and AI initiatives, supporting architectural decisions, defining technical standards, sharing knowledge, and collaborating with multidisciplinary teams. - Evaluate emerging technologies, frameworks, and approaches in Data Engineering, Artificial Intelligence, Machine Learning, and Generative AI, contributing to the continuous evolution of the organization’s analytical capabilities. - Develop and monitor business and technical performance indicators, translating analytical results into actionable insights related to efficiency, revenue, cost reduction, customer experience, and strategic decision-making. ### Requirements - Advanced proficiency in Python or R for data manipulation, automation, analytics, Machine Learning development, and implementation of production-ready solutions. - Strong experience with advanced SQL, including extraction, transformation, cleansing, integration, and analysis of large datasets while maintaining data quality and availability. - Proven experience with Machine Learning and predictive modeling, including the development, training, validation, optimization, and deployment of supervised and unsupervised models using tools such as Scikit-Learn, XGBoost, LightGBM, or similar frameworks. - Solid understanding of the end-to-end Machine Learning lifecycle, including model versioning, monitoring, reproducibility, automated pipelines, deployment, and production performance management. - Professional experience with Google Cloud Platform and large-scale data processing technologies such as Databricks, Apache Spark, and BigQuery, with an emphasis on scalability and performance. - Strong knowledge of statistics, statistical inference, mathematical modeling, and experimental design, including the ability to conduct and interpret A/B tests. - Experience designing scalable data architectures for data processing, storage, and consumption, applying sound practices for governance, quality, security, and observability. - Ability to connect technical and analytical outcomes to business objectives, transforming model performance and data insights into actionable recommendations and measurable value. - Experience defining and monitoring business and model performance metrics, evaluating outcomes such as efficiency gains, revenue growth, cost reduction, and improvements in customer experience. - Strong problem-solving skills and the ability to navigate complex analytical challenges, identify opportunities, and develop innovative solutions using Data, AI, and advanced Analytics. - Experience in the Healthcare sector is considered a plus. - Strong communication, collaboration, autonomy, and technical ownership, with the ability to work effectively across multidisciplinary teams and contribute to technical decision-making. ### Benefits - Opportunity to work with modern data, cloud, Analytics, AI, and Machine Learning technologies. - Hands-on exposure to Databricks, Spark, BigQuery, and Google Cloud Platform in large-scale data environments. - Opportunity to contribute to innovative AI, Machine Learning, and Generative AI initiatives with direct business impact. - Collaborative environment with multidisciplinary teams and opportunities for knowledge sharing and technical leadership. - Scope to influence data architecture, engineering standards, governance, and the evolution of analytical capabilities. - Opportunities for continuous learning and professional growth in a technology-focused environment. - Compensation and any additional benefits are determined by the hiring company and will be discussed during the recruitment process.