Senior LLMOps Engineer

Job description

We’re Heidi. We're building the future of healthcare by giving every clinician the earth's finest AI Care Partner. In just 18 months, our clinical AI products have absorbed the administrative chaos of 73 million patient visits. Today, we support over 2.5 million patient sessions a week across 190+ countries. Healthcare systems are failing us; clinicians spend more time on documentation than on patients, and the human connection that makes medicine worth practicing is eroding. Our mission is simple: double the world’s healthcare capacity and strengthen the human connection at its heart. We found product-market fit with a freemium medical scribe that clinicians love. Now, we're expanding. Every task a clinician hands to Heidi is a patient who feels more attended to, a health system unclogged, and a clinician who gets to be a clinician again. If you don’t choose easy and you want to build something way bigger than yourself then, choose the challenge, choose Heidi. The role This role sits in the model team, the researchers and engineers who train, deploy, and own the AI models behind every Heidi product. Those models run at serious scale; what they need now is a serious operational layer around them. You'll build it: full visibility into every deployment, a feedback flywheel that turns real clinician signals into better models, and per-model unit economics that tell us which models pay their way. We're looking for someone who has already built this at an AI company operating at or beyond our maturity, and can bring that playbook to Heidi. It's a hands-on senior engineering role: you'll design the data models, build the pipelines, dashboards, and agents, and own them in production. What you’ll do - Build the deployment health dashboard. Give the team live visibility into every model in production: health metrics, monitoring, and proactive incident alerting that catches problems before clinicians feel them. - Make every incident traceable. Build complete session-to-model lineage, so a degradation can be walked from affected sessions to affected user profiles to the exact model ID, scope of impact, and root cause in minutes. - Stand up the improvement flywheel. Start from Intercom tickets and qualitative CSAT feedback, and put AI agents to work identifying the exact session behind each piece of feedback. - Surface the full story. Retrieve the complete execution trace for every flagged session, bring it to the model team for review, and generate summaries that stakeholders outside engineering can act on. - Close the loop. Filter high-value feedback into training data, ship improved models, and monitor post-deployment performance so every release improves on the last. - Own per-model P&L. Measure revenue against inference cost for every model deployment, and turn model selection and deployment strategy into decisions backed by unit economics. - Raise our LLMOps bar. Bring the practices proven at the most mature AI companies (tracing, evaluation, model incident response) and make them how Heidi operates. - Partner across the model team. Work with the researchers and engineers behind our ASR, note generation, Evidence, and Dictate models so observability is built in, not bolted on. What you'll need - You've spent the last 2–3 years hands-on in an LLMOps role, building the observability, tracing, evaluation, and feedback systems around production LLMs and owning them through real incidents. - That experience comes from an AI company operating at or ahead of Heidi's maturity, most likely in the US or China, where LLMOps practice runs deepest. You know what great looks like, and you can build it here. - Proven ability to ship the systems this role owns: monitoring and alerting (Datadog or similar), distributed tracing across multi-step LLM pipelines, and session and event data models that hold up at scale. - Experience building with LLMs, not just operating them. You can put an agent to work triaging feedback and matching tickets to sessions. - Comfort joining cost and revenue data into per-model unit economics that leadership can act on. - Senior-level ownership: you take an ambiguous mandate, design the system, and run it in production. No PhD required; we care about what you've shipped. Nice to have - A broader engineering foundation before LLMOps: backend, data platform, or ML infrastructure. We're hiring senior, and range helps. - Experience wiring product feedback tools like Intercom into engineering systems. - Time in healthcare or another regulated, safety-critical domain. How we show up - Build for the next decade, not next quarter. Our targets are outrageous on purpose. The world's health doesn't have the luxury of incrementalism. - Lead, don't wait. We treat tomorrow's problems today. Sometimes we build what's needed before it's wanted, and we're fine with that. - Follow the evidence. Trust the patient. We pursue truth relentlessly. But when the subjective and objective disagree, we treat the patient, not the numbers. Ego is a comorbidity we can't afford. - Own the outcome. Everyone here carries the company. Raise problems with solutions, solve them end-to-end, and never be a bystander. - Ship, measure, go again. A button today, a workflow tomorrow. More iterations beat better planning. We're precise at pace, not reckless. - Live in clinicians' reality. Not the ideal workflow, the twenty-patients-before-lunch actual one. We build for exhausted humans, and we'd better be decent ones while we do it. Why Heidi? You’ll join a team focused on real-world impact over imaginary valuations and glossy PR. We live and breathe the challenges of modern health systems, and are laser-focused on exacting the change we’d like to see. We’re medicos, engineers, builders, and designers who’ve felt the moral and practical toll of what non-care feels like. True A-players progress extremely fast here. The nature of the scale-up game is demanding, but we value sustainable performance and mental health. You're trusted to perform, and you set your schedule. We operate on outcomes > inputs, not process theatre. We all take the bins out, metaphorically and literally. Building what we’re building isn’t always easy. But we didn’t choose easy, we chose to build something that actually matters. We hold ourselves to a higher standard because healthcare demands it. If you join Heidi, you recognise that the deeper question isn’t whether AI can solve the global healthcare crisis, but whose hands will shape it. The work is hard, but you will trust and admire the people you work beside, and rest easy knowing you’re doing the defining work of your career. We take care of you. We offer a $1,000 annual learning and development budget, a $150/month health and wellness allowance, a $500 home office budget, 26 weeks paid primary parental leave and 18 weeks paid secondary parental leave, fertility support up to $10,000, four weeks of work from anywhere per year, and serious equity.