Senior Manager, Data Platform Engineering

Job description

Prudential’s purpose is to be partners for every life and protectors for every future. Our purpose encourages everything we do by creating a culture in which diversity is celebrated and inclusion assured, for our people, customers, and partners. We provide a platform for our people to do their best work and make an impact to the business, and we support our people’s career ambitions. We pledge to make Prudential a place where you can Connect, Grow, and Succeed. Senior technical and people leader responsible for driving the end-to-end delivery of the Data Lake platform, Enterprise Data Model, data ingestion, and curated data services that power data & analytics, AI, regulatory, and business reporting needs. The role leads a multi-disciplinary team of data engineers and contingent specialists, while partnering closely with Data Governance, Enterprise Architecture, AI Engineering, Business Units, Group Technology, and external vendors to deliver trusted, governed, and scalable data assets. Key Responsibilities : 1. Team Leadership Lead, coach, and develop a team of data engineers, platform engineers, and contingent workers across onshore and offshore models, including direct line management of permanent staff and contractors. Provide day-to-day technical guidance, code/design reviews, and architectural direction to the team on Azure Databricks, ADF, Cosmos DB, MongoDB, SQL, Python/Scala, Qlik Replicate, Fivetran, Enterprise Kafka, DataStage, Informatica and ingestion frameworks. Drive performance management, career development planning, and succession planning for the team. Build a high-performing culture of accountability, quality, and continuous learning; identify upskilling/cross-training needs and nominate talent for recognition. Support hiring, interviewing, and onboarding of new team members and vendor resources. 2. Data Platform Engineering & Architecture Delivery Own the roadmap, design, build, and operations of the Data Lake platform, including raw, structured, EDM, and curated/serving layers on Azure Databricks and Azure Data Stack. Drive the data uplift programme and convergence/decommissioning initiatives (e.g., convergence, legacy data store remediation). Lead the design of scalable, reusable ingestion patterns (batch and near real-time) from core systems such as policy admin system, claims, marketing, and partner systems. Govern adoption of the group data model alignment (including SCD2 patterns, incremental processing, timestamp metadata) in line with Group/Regional standards. Champion engineering best practices: CI/CD, modular pipeline design, parameterised/config-driven frameworks, monitoring, and observability. 3. Data Governance, Quality & Compliance Support Partner with the Data Governance team to operationalise the Group Data Governance Operating Minimum Standards (data dictionary, CDE prioritisation, data quality, lineage, master data, data extraction). Provide technical stewardship for business and technical data lineage and support development/implementation of Data Quality rules and remediation. Act as the technical approver/gatekeeper for data lake access provisioning, PII access, schema-level entitlements, and data sharing (internal and external/RFP/vendor scenarios). Support responses to regulators (e.g., Bank Negara Malaysia / BNM) on data governance, manual submissions to core systems, PIA, and legacy remediation plans. 4. Business Continuity, Risk & Operational Resilience Serve as the department's Business Continuity Coordinator (BCC) for Data Management, owning the BCP, call tree, alternate-site/WFH arrangements, and annual BCP exercises. Ensure DR readiness, IT DR test participation, and recovery procedures for critical data platforms and pipelines. Manage incidents, root cause analysis, and post-mortem reviews for data platform and pipeline disruptions. 6. Vendor, Sourcing & Cost Management Lead RFP/RFI evaluations, technical scoring, vendor workshops, and engagement decisions for data-related solutions. Manage delivery partners and contingent workforce — including scope, SLAs, performance, and commercial governance. Drive resource sourcing for specialised roles (Data Engineer, Cosmos DBA, Data Modeler, Solution Architect) including JD preparation and interview leadership. 7. Documentation, Standards & Continuous Improvement Ensure functional, technical, operational, and architecture documentation is maintained for all supported platforms and pipelines. Embed security guidelines for confidential / PII / Restricted-Sensitive data in line with Group Security and Group Data Policy. Identify automation and optimisation opportunities to improve cost, performance, and time-to-insight. Job Requirements : Education & Experience Bachelor's Degree (Master's preferred) in Information Technology, Computer Science, Data/Information Engineering, or related discipline. 10+ years of progressive experience in data engineering / data platform / data management roles, with at least 4 years in a people leadership capacity managing teams of 5+ engineers. Proven track record of delivering enterprise-scale data platforms (data lake / lakehouse / EDW) end-to-end in a regulated industry — insurance, banking, or financial services strongly preferred. Technical Expertise Deep hands-on expertise in Azure Data Stack: Azure Databricks, Azure Data Factory (ADF), Azure Data Lake, Unity Catalog, Cosmos DB, and Power BI. Strong programming proficiency in Python, Scala, and SQL; familiarity with Spark optimisation and Delta Lake. Solid understanding of data architecture, dimensional modelling, EDM, MDM (e.g., PruMDM/Reltio), SCD2, data lake/lakehouse, data lineage, reconciliation, and CI/CD for data. Working knowledge of Data Governance & Data Quality tooling — Collibra and/or Informatica (CDGC, IDQ, EDC) is a strong advantage. Exposure to AI/ML data enablement (MLOps, feature stores, model-ready datasets) and integration with Salesforce Data Cloud is an added advantage. Familiarity with API/streaming integration patterns (Kafka, Event Hub, GraphQL, MongoDB connectors). Domain & Regulatory Knowledge Strong understanding of the insurance business domain (agency, customer, policy, claims, health, billing, product) and associated data flows. Awareness of BNM, PDPA, and Group regulatory expectations on data governance, privacy, and outsourcing. Leadership & Soft Skills Demonstrated ability to lead, mentor, and grow technical teams, including managing performance, succession, and capability building. Excellent stakeholder management — comfortable presenting to and influencing C-level audiences and Group/Regional counterparts. Strong executive communication skills: able to translate complex technical topics into business-impact language and concise executive briefs. Highly proactive, self-motivated, and accountable; able to cope with challenging delivery timelines and concurrent priorities. Strong analytical, problem-solving, and decision-making skills with sound commercial judgement. Excellent interpersonal and written/oral communication in English. Prudential is an equal opportunity employer. We provide equality of opportunity of benefits for all who apply and who perform work for our organisation irrespective of sex, race, age, ethnic origin, educational, social and cultural background, marital status, pregnancy and maternity, religion or belief, disability or part-time / fixed-term work, or any other status protected by applicable law. We encourage the same standards from our recruitment and third-party suppliers taking into account the context of grade, job and location. We also allow for reasonable adjustments to support people with individual physical or mental health requirements.