Software Engineer – Data Platform

Job description

Job Description This is a software engineering role focused on building production-grade data platform components. You will design and operate services, APIs, and data pipelines in a CI/CD-driven environment. This is NOT a notebook-driven or analytics-focused role. Job Description: Design, implement, and maintain secure, scalable and fault-tolerant data ingestion pipelines from a wide variety of source systems, including SAP, Salesforce, SharePoint, APIs, and (legacy) manufacturing platforms. Build and enhance production-grade metadata-driven data handling components and inbound-/outbound-interfaces. Own production data interfaces end-to-end. Develop reusable, version-controlled platform components, APIs, or data interfaces, following software engineering best practices (CI/CD, automated testing, version control, monitoring). Contributes to a global data engineering team delivering to all major business domains. Establish and maintain holistic data quality management, monitoring and reporting. Collaborates across business IT, cybersecurity, infrastructure, and architecture teams to ensure secure and sustainable delivery. Main Tasks: Design and implement fault-tolerant, observable data pipelines, including logging, alerting, and failure handling strategies. Build and maintain Python- or Scala-based extraction services (e.g., Debezium Server, custom APIs, rclone) Implement CDC, delta, and event-based patterns. Build and enhance push-based HTTP, Kerberos-authenticated custom data lake transfer delivery component. Integrate with systems such as Salesforce, SharePoint, and other API- or file-based endpoints. Establish, operate and troubleshoot extraction from SAP using tools like Theobald Extract Universal. Establish and maintain a web-accessible data catalog application as part of the data platform, focusing on backend services, metadata management, and integration with platform components. Integrate dataset discoverability, preview/exploration options, and lineage information using Unity Catalog as a backend metadata system. Design and implement structured access request workflows including request submission, approval chains, audit trail, and enablement triggers. Perform design reviews with Cybersecurity. Ensure documentation and compliance for all interfaces and data ingress points. Manage audit and traceability requirements. Collaborate closely with IT and business users to translate requirements into scalable technical patterns. Serve as technical escalation point for complex source integration. Define and implement a multi-layered data quality framework, including unit-level, integration-level, and cross-pipeline validation rules. Establish centralized and version-controlled storage of DQ rules, with integration into orchestration and CI/CD pipelines. Implement automatic DQ monitoring with severity levels (Critical, High, Medium, Low) and enable flagging, filtering, and quarantining logic at relevant stages of the pipeline. Collaborate with source system owners and business stakeholders to define meaningful and actionable DQ thresholds. Degree in Computer Science, Data Engineering, or a related field. 5–8 years in software development, with hands-on experience with inbound- and outbound data exchange interfaces, using a variety of technologies, such as APIs, CDC, managing endpoints. Hands-on experience with Scala / Java, Python, and advanced SQL, using Git for collaborative development and CI/CD methodologies. Experience building robust ingestion pipelines and ETL-jobs in a software engineering context. Track record with Azure or other hyperscaler resource management. Familiarity with data lake housing and data warehousing, Databricks and Azure knowledge is a plus. Comfortable working across geographies and time zones; collaborates effectively with global teams and enterprise stakeholders. Candidates with a strong backend/software engineering background (e.g. Java, Scala, distributed systems) who want to move into data platform engineering are highly encouraged to apply. NOT a fit if your experience is primarily: ▪ Notebook-based data processing (e.g. Jupyter, ad-hoc PySpark) without production deployment ▪ Dashboarding / analytics / BI tools ▪ SQL-only roles without software engineering experience The well-being of our employees is important to us. That's why we offer exciting career prospects and support you in achieving a good work-life balance with additional benefits such as: Training opportunities Mobile and flexible working models Sabbaticals and much more... Sounds interesting for you? Click here to find out more. Diversity, Inclusion & Belonging are important to us and make our company strong and successful. We offer equal opportunities to everyone - regardless of age, gender, nationality, cultural background, disability, religion, ideology or sexual orientation. Ready to drive with Continental? Take the first step and fill in the online application. Continental develops pioneering technologies and services for sustainable and connected mobility of people and their goods. Founded in 1871, the technology company offers safe, efficient, intelligent, and affordable solutions for vehicles, machines, traffic and transportation. In 2023, Continental generated sales of €41.4 billion and currently employs around 200,000 people in 56 countries and markets. Guided by the vision of being the customer's first choice for material-driven solutions, the ContiTech group sector focuses on development competence and material expertise for products and systems made of rubber, plastics, metal, and fabrics. These can also be equipped with electronic components in order to optimize them functionally for individual services. ContiTech's industrial growth areas are primarily in the areas of energy, agriculture, construction, and surfaces. In addition, ContiTech serves the automotive and transportation industries as well as rail transport. The IT Digital and Data Services Competence Center of ContiTech caters to all the Business Areas in ContiTech and responsible among other on areas of Data & Analytics, Web and Mobile Software Development and AI The team for Data services specializes in all platforms, business applications and products in the domain of data and analytics, covering the entire spectrum including AI, machine learning, data science, data analysis, reporting and dashboarding.