Lead Engineer, EDT Data & Analytics Cloud Engineering

Job description

Work Flexibility: Hybrid Lead Engineer, EDT Data & Analytics Cloud Engineering The Lead Cloud Platform Engineer supports the operational excellence, security, and reliability of the enterprise data & analytics platform on Azure and Databricks. The role combines platform operations, DevOps automation, and Infrastructure as Code across the core stack - Azure Databricks, Synapse, ADLS Gen2, Azure SQL, Azure Data Factory, Key Vault, and Unity Catalog - with a reliability-engineering mindset: automate repeatable work, apply security best practices by default, and help keep the platform highly available for global data engineering and analytics teams. What you will do: ​​ DevOps Pipeline Management & CI/CD Design, build, test, and operate Azure DevOps CI/CD pipelines that package code, run automated quality checks, deploy infrastructure and data workloads, and promote releases across development, testing, and production environments. Own day-to-day pipeline execution and release readiness by monitoring runs, diagnosing failed builds and deployments, correcting pipeline definitions, and improving rollback, approval, and promotion controls. Infrastructure as Code & Environment Provisioning Write, review, and maintain Terraform modules for repeatable provisioning of Azure and Databricks platform components; test changes through plan and deployment cycles and keep modules versioned and reusable. Provision and configure workspaces, compute, storage, networking, private connectivity, identities, and access through automated IaC processes; detect configuration drift and implement corrective changes. Platform Security & Access Management Configure and troubleshoot RBAC, service principals, managed identities, secrets, storage permissions, cluster policies, and Unity Catalog grants across Azure Data Factory, Databricks, Synapse, Azure SQL, ADLS Gen2, and Key Vault. Apply and verify least-privilege access, identity integration, and secrets-management controls through hands-on configuration reviews, access testing, and remediation of security findings. Platform Operations & Reliability Engineering Build and tune monitoring, alerts, dashboards, and operational runbooks; participate directly in incident response, perform root-cause analysis, and implement preventive automation and reliability fixes. Monitor integration runtimes, clusters, jobs, pipelines, and platform services; inspect logs and metrics, reproduce failures, resolve runtime and performance issues, and validate service recovery against SLAs. Data Engineering Operations Work directly with Data Platform and Data Product teams to deploy and operationalize ingestion pipelines, orchestration, Databricks jobs, and supporting platform services; provide hands-on support through production rollout. Collaboration Translate stakeholders into implementable technical changes, contribute code and configuration, lead technical working sessions, and document the resulting solution and operational handoff. What you need: Bachelor’s or master’s degree in computer science, Engineering, Information Systems, Business, or a related technical discipline. 5-7 years of experience designing, engineering, and supporting cloud-based data and analytics platforms, data warehouses, and modern data ecosystems. Strong hands-on expertise with Azure data services, including Azure Databricks, Synapse Analytics, ADLS Gen2, Azure SQL, Azure Data Factory, Key Vault, and Unity Catalog. Proven experience implementing platform security controls, governance frameworks, and Azure data platform best practices to ensure secure, scalable, and compliant solutions. Proficiency in Infrastructure as Code (IaC) using Terraform and Azure DevOps CI/CD pipelines, with experience automating deployments across multiple environments. Demonstrated ability to monitor, optimize, and troubleshoot cloud data platforms, including performance tuning, alerting, incident management, and root cause analysis. Excellent analytical, problem-solving, and communication skills, with the ability to collaborate effectively across engineering, operations, security, and business stakeholders. Proactive mindset with a track record of driving automation, process optimization, operational improvements, and scalable platform enhancements that increase efficiency and reliability. Cloud Engineering certifications on Azure/Databricks platform is a plus. Travel Percentage: None