Data Engineering Lead

Job description

**Location:** London (UK) **Department:** Engineering **Hours:** Full-time ### About YLD: YLD is a software engineering and product design consultancy, founded in 2013, headquartered in London, with delivery teams in Lisbon and Porto. We work with engineering and product leaders who know what good looks like and are frustrated by the gap between strategy and execution. We embed senior engineers and designers inside client teams, working in their codebase, their tooling, their delivery rhythm. AI runs through how our teams work, and we help clients adopt it into their own workflows too. ### About the role: We're building out data as a first-class practice at YLD, and we need someone in London who can be the credible technical voice in front of clients and prospects, not just describing what we could do, but backing it with real delivery experience. This is a senior role, reporting to our Head of Engineering. You'll sit alongside our commercial and client services leadership in meetings where technical credibility matters, help shape and win new engagements, and stay close enough to client work to guide it and step in yourself when an engagement needs you. Alongside, we expect you to build our external reputation in data through events, writing, and the wider community. ### About you: You've been a strong data engineer or data lead for years, and you're comfortable in front of a client as well as in a pull request. You know what good data work looks like well enough to spot it, or its absence, in the first ten minutes of a client conversation. You've probably led a team before. Consultancy background is a strong plus, but not essential if you've got real client or stakeholder exposure elsewhere. You don't need an existing public profile in the data community, but you'll need the appetite to build one and the technical depth to make it credible when you do. ### Your day-to-day responsibilities will look like this: **Hands-on delivery (core, from day one)** - Doing hands-on data engineering work directly, pipelines, transformation layers, models, and the infrastructure around them, not just guiding others - Adapting to different client contexts: legacy warehouse migrations, greenfield lakehouses, transformation layers that need rescuing - Guiding the wider team and unblocking hard technical problems as your remit grows beyond your own delivery - Defining and evolving our internal standards for data work: testing, documentation, project structure, code review - Mentoring and growing data engineers across the company **Client and commercial (core, from day one)** - Joining client and prospect meetings as the technical authority on data, alongside our client services and commercial leads - Shaping proposals and pitches, translating client problems into a credible data approach and a defensible scope - Acting as the escalation point on live data engagements when technical judgement calls need weight behind them - Travelling to client sites for workshops, discovery sessions, and in-person relationship building, and representing YLD at external data events **Practice, talent, and external profile (growth, over the first year)** - Owning the growth of YLD's data practice: capability, reputation, and headcount over time - Contributing to attracting and hiring strong data engineers into YLD - Representing YLD at data events and in the wider community as your standing in the market builds - Supporting our sales team so that data becomes something we can proactively sell, not just respond to ### You'll have the following skills and experience: **Leadership and commercial** - Consulting or agency experience is a strong plus, but real client or stakeholder-facing exposure elsewhere is acceptable - A track record of technical leadership: leading a team or function - Comfortable presenting to senior client stakeholders and holding your own under commercial pressure - Cross-functional fluency: you translate engineering constraints into business terms and negotiate realistic commitments - Appetite to build a public presence in the data community over time, even if you don't have one yet **Technical** Not theoretical, evidenced by things you've actually shipped, and deep enough to set the bar for a team and answer a sceptical client: - SQL as engineering: read and review SQL that performs at scale, and understand query planning, engine quirks, and how materialisation choices affect cost and performance well enough to guide a team's decisions - Python for data: built and maintained production data systems in typed, testable Python, not just notebooks, and can set that standard for others - Data modelling: made deliberate choices between dimensional, Data Vault, normalised and denormalised designs, and can explain the trade-offs in flexibility, query performance, and maintainability to both engineers and clients - Transformation architecture: design transformations that are idempotent, incremental, and dependency-aware. You think in DAGs, not scripts - Pipeline design: weigh batch, streaming, and micro-batch trade-offs against latency, complexity, cost, and reprocessability, and pick the right approach for the problem - Data testing: know what to catch at build time (schema contracts, assertions, transformation logic) versus defer to observability, and can make that call for a team - Data observability: treat data reliability like site reliability, with measurable indicators, alerting, incident response, and root cause analysis - CI/CD for data: version, test, and deploy pipelines like software, with environment promotion, rollback strategies, and infrastructure as code - Data governance: implemented lineage, cataloguing, sensitive data classification, or access control, and can hold a client to account on making data auditable and secure - Cost-conscious: optimised warehouse spend, storage strategies, or job efficiency, and treat compute as a resource to manage, not ignore - Platform fluency: worked across orchestration (Airflow, Dagster), warehouses (Snowflake, BigQuery, Databricks), ingestion (Fivetran, Airbyte, custom), and transformation (dbt, Spark) - AI Native: experienced in agentic coding workflows (context, harness, loop, and graph engineering), and able to embed AI Engineering constructs into pipelines (RAG, semantic search, evals, guardrails, etc) ## **You should apply if:** - Confident and credible in front of clients and prospects, without needing a script, and comfortable holding a firm technical or commercial line when a client or prospect pushes back - Still able to hold up in a deep technical review today, architecture, data modelling, pipeline design, not just talk about decisions you made a few years ago - Self-motivated, and want to own building a practice, not just execute a workstream someone else has defined, and comfortable without a fully defined remit from day one - Able to work well with commercial and client services colleagues who aren't technical themselves, translating for them without talking down - Genuinely energised by variety: different clients, different stacks, different problems, rather than depth in one domain - Happy travelling regularly, client sites, workshops, industry events, wherever the work and the relationship-building actually happens - Looking for a role that combines hands-on delivery with client-facing exposure and external visibility (speaking, writing, community), not just one of these ### **Benefits you’ll receive:** - Company Private Health care (currently provided by Vitality) - Enhanced fully paid maternity and paternity leave for up to 6 months - Company's Pension Scheme - 25 days annual holiday (excluding Public Holidays) - £2000 annual allowance for Training/Conferences - £300 annual allowance for additional hardware - Mental Health support - Bonus (depending on Company performance and results) - Company laptop - Generous referral schemes ### Our typical Recruitment Process looks like this: - **Screening call** with the Talent team (45-60 mins) - **Technical conversation** with our Head of Engineering and a senior data specialist (60 mins) - **Scenario exercise** with our Head of Engineering, Head of Client Services and COO (60–90 mins) [**Get a glimpse of what it's like working at YLD! 🤳**](https://www.instagram.com/yld.io/) [**Check out our LinkedIn page**](https://www.linkedin.com/company/yld/) ### We live and breathe our values, and know you will too: - Growing every day - Including everyone - Relationships built on honesty and ethics - Inspiring solutions - Winning together We’re an equal-opportunity employer and value diversity in all its forms. We do not discriminate based on race, religion, colour, national origin, gender, sexual orientation, pregnancy or maternity, age, marital status, or disability. We also offer a remote-first working environment, with flexible working and work–life balance as standard for all employees.