Data Engineer, Databricks

Job description

The Data Engineer is responsible for the design, development and support of the Data Products Cuscal creates for both internal and client facing use cases by primarily working on sourcing, transforming, securing and monitoring data for the creation of products using Cuscal’s data lake and analytics systems. The data engineer will be part of a scrum team and actively engages in both discovery and delivery activities and as such will have direct interaction with our clients, specifically their product, engineering and analytics personnel. The data engineer will work in partnership with our Data Practice to deliver solutions that meet our organisational standards and best practices To suceed in this role you will: Participate in discovery workshops to understand internal and client problems Work with the product owner to design solutions that address identified pain points Employ Cuscal infrastructure for extraction, transformation, and loading (ETL) of data from a wide variety of data sources using ‘Big Data’ technologies Create and maintain Data pipelines in an efficient manner Maintain, troubleshoot, and improve the workflow and job scheduling of ETL processes Assist in the design & development of analytic solutions. Mostly in Databricks SQL Analytics &/or PowerBI Manage and process complex, large data for analysis (e.g. hierarchies of datasets, inter-relationships, partitions, views, etc.) Maintaining a high level of system availability, system monitoring, fault diagnosis and change implementation and review Work with our Data Practice to support the tracking, analysis and review of trends in commonly occurring support issues and then embed identified risk management strategies. Maintain and support applications in line with approved standards and adjust in alignment with Enterprise Architecture Respond to incidents and manage systems availability (change, incident and problem management) Take part in 24 x 7 on call support roster, as required. You'll stand out if you have: 5+ years in Data Engineering, ideally in Payments Strong knowledge in Databricks, Datalake and Data Warehousing techniques. Experience in one or more programming languages, Python preferred Experience with Oracle and/or SQL databases and writing SQL. · Knowledge and understanding of Databricks tuning and functions Experience in Analytics; Databricks SQL Analytics &/or PowerBI preferred. · Experience in scripting & Infrastructure as Code; eg: PowerShell, Terraform · Experience in the financial services or payments sector · Experience working within PCI-DSS compliant environment · Knowledge of project delivery techniques and methodologies · Knowledge of Software Delivery Lifecycle techniques and methodologies · Knowledge of Secure Development lifecycle (SDLC) · Sound knowledge of ITIL standards. Why Cuscal? At Cuscal, you’ll experience interesting work that’s transforming. You get the security of an established organisation and the energy of a company committed to innovation. For 60 years, we’ve set the standard in payments, and now we’re preparing to redefine it for the future. You’ll also enjoy a range of benefits, including: Recognition and growth : Celebrate achievements through IGNITE and grow with tailored development opportunities Wellbeing focus: We support your physical, mental and financial health with holistic initiatives and access to discounts via Cuscal Advantage Inhouse Barista: for whenever you feel like your favourite hot beverage If you’re excited about this opportunity, we’d love to explore how you can contribute to our vision. Applications may be reviewed as they’re received, so don’t wait - apply now. Cuscal is an equal opportunity employer committed to creating a diverse, inclusive and barrier-free workplace. We welcome applications from Aboriginal and Torres Strait Islander peoples, people living with disability, LGBTQIA+ communities and individuals from culturally diverse backgrounds. #LI-NL Forward with Cuscal At Cuscal, our Commercial and Business teams are the backbone of our organisation. From supporting clients and driving growth to ensuring compliance, managing finances, and fostering a positive workplace culture, our people play a vital role in achieving our commercial objectives while staying true to our values. If you’re looking for a role where your skills and ideas make a real impact, join us to help shape the future of payments in Australia.