Software Engineer II (Python) - Supply & Demand

Job description

At Kpler, we are dedicated to helping our clients navigate complex markets with ease. By simplifying global trade information and providing valuable insights, we empower organisations to make informed decisions in commodities, energy, and maritime sectors. Since our founding in 2014, we have focused on delivering top-tier intelligence through user-friendly platforms. Our team of over 700 experts from 35+ countries works tirelessly to transform intricate data into actionable strategies, ensuring our clients stay ahead in a dynamic market landscape. Join us to leverage cutting-edge innovation for impactful results and experience unparalleled support on your journey to success. Join Kpler’s Commodities Tribe as our next **Software Engineer II** and help shape the future of our Supply & Demand products. Our Supply & Demand platform transforms complex global commodity data-from production sites, trade flows, imports, exports, refinery activity, consumption, and stock changes-into trusted market intelligence used by our clients around the world. Historically focused on crude oil, our team is now evolving the platform to support a much broader range of commodities, including refined products, biofuels, metals, and beyond. This is a major engineering transformation: building a scalable, reusable platform that can support new commodity markets without creating separate solutions for each product. As aSoftware Engineer II, you will contribute to the development and operation of the systems behind our Supply & Demand products. You will own meaningful technical areas, delivering reliable solutions across data pipelines, backend services, APIs, and data-quality systems. You will work closely with experienced engineers, Product, Research, and Data teams to solve complex problems where domain knowledge, data challenges, and engineering decisions come together. This role is ideal for an engineer who enjoys taking ownership, working with complex data, and growing their technical impact while contributing to products used by customers globally. ## ### Your future position: - Own a commodity balance end-to-end: take half-defined blending rules through computation, validation and a reliable production release on the Airflow pipeline, including the on-call afterward. - Lead the resolution of critical bugs and incidents in the crew's pipelines, databases and distribution APIs; turn each incident into a runbook entry and a prevention. - Extend and harden the anomaly-detection / data-quality gate (Prophet-based) so bad numbers are caught before publication, reducing false positives and time-to-detect. - Help generalise the S&D data model and the shared ETL framework so adding a commodity is an extension, not a fork of the engine. - Design and incrementally deliver well-scoped milestones for crew projects; write and review ADRs and participate actively in design reviews. - Build and evolve the data-distribution APIs (Oil and Grains TypeScript services) consumed by the webapp and by external clients. - Onboard and mentor junior (P2, P1) engineers joining the crew: pairing, reviewing, and growing their independence. - Partner with Research, Product and Data to turn half-defined data needs into well-scoped, deliverable engineering tasks. - Identify and close gaps in observability and monitoring across the crew's DAGs and services, and manage escalations for the crew. - Participate in hiring loops as a calibrated interviewer. ### Experience & Background: **Essential:** - 2+ years of experience as a software engineer, with experience delivering production systems. - Experience owning a non-trivial feature or service end-to-end in production (design, ship, monitor, and fix), not just delivered tickets. - Strong Python for data-heavy work (e.g. pandas/polars, SQLAlchemy or equivalent ORM, a relational database with real schema evolution). - Comfortable debugging and operating production data pipelines, and has carried on-call or first-responder duty for systems they built. - Has taken a half-defined problem and shaped it into a delivered solution without waiting to be told the route. **Desirable:** - Experience with workflow orchestration (Airflow / Astronomer or similar) and batch/ETL data pipelines. - TypeScript/Node API experience (NestJS, Express, tsoa, TypeORM) for the distribution-API surface. - Exposure to commodities, energy, or any domain with messy, multi-provider data that must be reconciled. - Has mentored a junior engineer or onboarded a new joiner. - Familiarity with anomaly detection / forecasting (Prophet, scikit-learn) or data-quality tooling. **Behavioural Competencies:** - You take responsibility for delivering outcomes, not just completing individual tasks. You follow through on your work and actively look for ways to improve the systems you contribute to. - You understand that delivering software includes testing, monitoring, supporting production systems, and learning from incidents. - You are comfortable working through complex problems, breaking down ambiguity, and collaborating with others to find practical solutions. - You work effectively with engineers, Product, Research, and Data teams, sharing context and building solutions together. - You actively seek feedback, learn from others, and continuously develop your technical skills and understanding of the domain. - You care about building reliable, maintainable software and consider the impact of your technical decisions on customers and teammates.