Data Engineer

As our new Data Engineer, you will join a collaborative and skilled organization working with data at scale to enable automation of real business cases through data-driven and Machine Learning-based intelligence. We are hiring two Data Engineers for two closely collaborating teams with different focus areas: the Data Science team , working directly with forecasting, analytics, and ML-based decision support the Data Streaming Platform (DSP) team, a newly formed team building and operating a central platform for real-time and event-driven data across the organization We work in a Scrum setup and have in recent years been very successful in forecasting both electricity prices and electricity consumption, while at the same time scaling our real-time data capabilities. Your work will have direct and measurable impact . What will you do? Design, build, and operate batch and real-time data pipelines using Azure, Databricks, and Python Ingest and process external and internal data such as weather data, market prices, grid telemetry, and event streams Work with streaming technologies including Kafka, Spark Structured Streaming, and Flink Ensure high data quality through validation, monitoring, and lineage Develop and improve data workflows, standards, and reliability in production systems Dependent on your area of expertise and interest, you will work in either the Data Science- or Data Streaming Platform team Data Science team focus: Enable forecasting, analytics, and ML use cases with high-quality, well-modelled data Collaborate closely with Data Scientists, ML Engineers, and other engineering teams DSP team focus: Build Kafka based central streaming data platform from scratch to enable traders, analysts and asset managers to publish and consume live and historical data at scale. Operate and continuously enhance the components once the platform is in production. Location Katowice or remotely in Poland Who are you? The focus varies depending on which role you are interested in. You don’t need check all the boxes, but generally you should be experienced in: Strong Python and SQL skills Practical experience with cloud data platforms (Azure, Databricks) and with different data storage solutions, including data lakes, lake houses and analytical databases Hands-on experience with streaming systems (Kafka, Spark or Flink) Familiarity with Airflow, CI/CD, DevOps practices Experience with Docker, Kubernetes, and Infrastructure-as-Code Understanding of data contracts, time-series data, and event-driven systems Fluency in English and interest in the energy domain Why Join Us? The journey to make real-time data truly scalable in BA Markets starts now, Be part of it! Build data solutions powering real-time trading and forecasting Work with modern data technologies and real production systems Strong work-life balance and purpose-driven culture focused on a fossil-free future Our offer Good remuneration, a challenging and international work environment, and the possibility to work with some of the best in the field. You will be working in interdisciplinary teams and you can always count on support from committed colleagues. We offer attractive employment conditions and opportunities for personal and professional development. More Information We welcome your application in English. We kindly request that you do not send applications by any means other than via our website as we cannot guarantee that we will be able to process applications that are not made via our website. For more information about the recruitment process you are welcome to contact our recruiter Marta Jura via . At Vattenfall we are convinced that diversity contributes to build a more profitable and attractive company and we strive to be good role model regarding diversity. Vattenfall works actively for all employees to have the same opportunities and rights regardless of gender, ethnicity, age, transgender identity or expression, religion or other belief, disability or sexual orientation. The security of Vattenfall and its employees is essential. For that reason, a pre-employment screening will be part of your recruitment process. The screening is based on the role you will fulfill within Vattenfall. The screening will be performed by a third party, DISA Global Solutions AB. We look forward to receiving your application! Vattenfall is a European energy company with approximately 21 000 employees. For more than 100 years we have electrified industries, supplied energy to people’s homes and modernized our way of living through innovation and cooperation. We are looking for talented individuals who, in addition to their passion for their own role, also have strong team spirit and want to contribute to supporting a meaningful corporate mission.