Lead Data Engineer (DBT, Databricks, Azure)

Job description

**Job Title:** Lead Data Engineer (DBT, Databricks, Azure) **Key Skills:** Python, REST APIs, Azure Databricks, DBT, SQL, Terraform, Data Engineering, Data Modeling, CI/CD, Team Leadership **Experience:** 8+ years of experience. **Location:** Open to candidates across LATAM. **At Coforge, we are looking for a Lead Data Engineer (#22145) with the following profile.** ### Responsibilities • Lead the RTS delivery squad and own the end-to-end delivery of strategic data products within a modern cloud data platform. • Design and oversee the architecture of low-latency, API-driven data solutions, from backend data models through serving layers and API consumption. • Build and maintain scalable backend services and REST APIs using Python and object-oriented programming principles. • Design, develop, and optimize data pipelines using Azure Databricks, DBT, SQL, and cloud-native technologies. • Own the architecture and implementation of curated data layers, ensuring high availability, scalability, and performance. • Evaluate, select, and implement technologies for low-latency data serving and API consumption. • Lead infrastructure deployment and management using Terraform and Infrastructure as Code (IaC) practices. • Mentor and develop a team of data and machine learning engineers, providing technical guidance, coaching, and career development support. • Conduct code reviews and establish engineering standards, development best practices, and architectural guidelines. • Collaborate with Product, Engineering, Business, and Project stakeholders to define technical solutions and delivery roadmaps. • Ensure data quality, observability, monitoring, alerting, and SLA compliance across all data products and services. • Drive CI/CD adoption and oversee release management processes for pipelines, services, and platform components. • Act as the primary technical escalation point for troubleshooting complex production issues and ensuring long-term platform stability. • Maintain comprehensive technical documentation, architectural decisions, and implementation standards. ### Mandatory Requirements • 8+ years of experience in Data Engineering, Software Engineering, or related technical roles. • At least 3 years of experience leading engineering teams, including senior, mid-level, and junior professionals. • Strong backend development experience with proven delivery of production-grade REST APIs and backend services. • Expert-level programming skills in Python, including strong object-oriented programming (OOP) principles. • Extensive experience with PyTest, unit testing, automated testing frameworks, and software quality practices. • Strong hands-on experience with Azure Databricks and DBT. • Advanced SQL skills and experience building scalable data solutions. • Deep understanding of data modeling, dimensional modeling, and consumption-oriented data architectures. • Extensive experience with Terraform and Infrastructure as Code (IaC) deployments. • Experience designing and delivering low-latency data products, serving layers, or API-driven data solutions. • Strong knowledge of CI/CD pipelines, Git-based version control, and modern software engineering practices. • Proven ability to lead technical delivery while remaining hands-on with architecture, development, and code reviews. • Excellent communication and stakeholder management skills, with the ability to work effectively across technical and business teams. ### Preferred Requirements • Experience within insurance, financial services, or other highly regulated industries. • Knowledge of insurance concepts such as policy administration, claims, underwriting, and customer data. • Experience with Snowflake and Snowflake-to-Databricks integration or migration patterns. • Hands-on experience with Azure AI Search, Lakebase, feature stores, or similar low-latency serving technologies. • Experience with streaming and real-time data technologies such as Kafka, Event Hubs, or Delta Live Tables. • Experience implementing data governance, security, and compliance frameworks. • Experience managing large-scale DBT environments and terabyte-scale data pipelines. • Experience leading cloud migration initiatives, particularly AWS-to-Azure migrations. • Familiarity with machine learning platforms and collaboration with ML engineering teams. **Published on:** 25-08-2026 At Coforge, we hire professionals solely based on their skills and qualifications and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.