Senior Technical Lead
HCL Technologies Limited
Posted Nov 16, 2026
Apply by Dec 22, 2026
- Other
Job description
Job Summary Senior Data Engineer Role purpose We are looking for an experienced hands-on Senior Data Engineer to support the migration of existing PBWM PEGA-related data pipelines away from legacy SAS and Teradata processing and onto the AWS-based data platform. The engineer will pick up work already progressed earlier this year, complete the remaining SAS-to-PySpark migration, and implement the producer-side changes required to supply data into the agreed PEGA integration pattern. Key Responsibilities Key responsibilities Understand the existing SAS/Teradata pipelines supporting PBWM PEGA data provisioning and baseline the remaining migration scope. Convert remaining SAS transformation and business logic into production-quality PySpark. Develop and enhance batch data pipelines using Python/PySpark and AWS, particularly Glue and S3. Maintain equivalent business logic and data structures where appropriate to support a controlled migration. Reconcile PySpark outputs against the existing SAS/Teradata processes and support parallel-run/data validation. Work with PBWM, EDT Simplification, architecture and downstream PEGA engineering teams to resolve integration dependencies. Support testing, deployment, release and productionisation of the migrated pipelines. Document the solution and provide an effective handover to the permanent engineering team. Skill Requirements Essential skills Strong Python and PySpark AWS data engineering, particularly S3 and AWS Glue SQL and complex data transformation Experience migrating legacy ETL/data-processing workloads to cloud platforms Experience translating or reverse-engineering SAS workloads into Python/PySpark Batch data pipeline development Data reconciliation, testing and validation Git-based development and CI/CD Experience working in controlled enterprise production environments Desirable SAS development experience Snowflake, Airflow/MWAA or EMR Teradata migration experience PEGA / Customer Decision Hub exposure Previous NatWest engineering experience would be advantageous PEGA expertise isn't essential. The primary requirement is a strong Data Engineer who can take unfamiliar SAS code, understand the existing processing, reproduce it safely in PySpark, validate the results and take the solution through to production. Other Requirements