Senior Engineering Manager, Observability Engineering

Job description

## Senior Engineering Manager, Observability Engineering Temporal is an open source programming model that can simplify code, make applications more reliable, and help developers focus on the important things like delivering features faster. We are on a mission to be the reliable foundation of every developer's toolbox, and are building the team that will make that happen. Our values guide us — they are present in how we show up, make decisions, and work together to make an impact. We're curious, driven, collaborative, genuine and humble. Temporal is growing and we are looking for those who share our values, challenge 'standard' thinking, and want to influence our future. If you have a passion for improving the developer experience, building world-class open-source software and communities, and want to be a part of our amazing team, we'd love to hear from you! Companies at the frontier of the AI revolution run on Temporal. OpenAI runs on Temporal, handling millions of requests. Cursor runs its cloud coding agents on Temporal at over 50 million actions a day across 7M+ workflows. Lovable, Abridge, and Hebbia build their agents on it too. In the last year alone, AI-native companies executed 1.86 trillion actions on Temporal Cloud, and the curve is still bending upwards. Backed by a recent $550M Series E at a $12.55B valuation, we are building the durable execution layer the agentic era depends on. Observability is how we and our customers know that layer is holding. This role owns both halves of that problem. **Internally**, you own the telemetry platform our engineers use to run a multi-tenant service that some of the largest companies in the world rely on. **Externally**, observability is a product surface. Customers debug their own workflows through what we expose, and the quality of that experience is a key feature of the platform. We are looking for a leader who has run observability at scale, treats telemetry cost and cardinality as first-class engineering constraints, and has the operational rigor to be accountable for live-site outcomes on a platform where reliability *is* the product. ## **Responsibilities** - **Observability roadmap:** Own the team’s technical roadmap and delivery, aligned with broader engineering priorities. Evaluate build, buy, and open-source options, balancing reliability, customer needs, and cost**.** - **Technical leadership:** Lead, hire, and grow a high-ownership team of engineers. Guide technical decisions through design and code reviews. Coach engineers, help them grow, and clear the friction that slows them down. - **Platform ownership:** Own the ingestion, storage, query, and visualization path for high-cardinality telemetry across a multi-tenant, multi-cloud fleet. Enable self-serve capabilities to support a growing Engineering organization. - **Operational excellence:** Own alerting and incident response tooling end to end. Drive the detection quality, routing, and clear, actionable runbooks that make on-call sustainable, and turn incident reviews into lasting reliability improvements. - **Customer-facing observability:** Partner with Product and SDK teams on the telemetry Temporal exposes to customers: metrics, traces, and workflow-level diagnostics. Work directly with design-partner customers and integrate what you learn into the roadmap. - **Cost, capacity & economics:** Own the unit economics of observability: cost per ingested series, per query, per namespace. Set cardinality budgets, drive consolidation where tool sprawl has crept in, and negotiate and manage vendor relationships against clear value criteria. - **Partner with Engineering, Product, Security, and Support** to resolve competing priorities and deliver shared outcomes. Explain technical tradeoffs and risks clearly to senior leaders and non-specialist partners. ## **Qualifications** - Proven experience leading engineering teams that build and operate observability, monitoring, or telemetry platforms at scale, with strong operational practices. - Demonstrated success leading engineering teams that build and operate production observability platforms, including hiring, coaching, performance management, and delivery. - 15+ years in software and/or infrastructure engineering, including 7+ years of people management, with demonstrated ownership of delivery and live-site outcomes. - Experience hiring, leveling, and performance management. - Strong distributed-systems expertise with the hands-on judgment to guide architecture and execution rather than guide it from a distance. - Experience operating multi-tenant infrastructure that other people's production workloads depend on. - Bachelor's degree in Computer Science or related field, or equivalent practical experience; advanced degree a plus. - Excellent communication skills, with the ability to partner across engineering, product, and leadership and translate customer feedback into a roadmap. ## **Required Skills** - Strong leadership and coaching; ability to grow engineers and build a healthy, accountable, high-ownership team, including through reorgs, acquisitions, or platform migrations. - Set priorities and deliver incremental improvements with incomplete information. Make tradeoffs explicit and adapt plans as customer and operational needs change. - Command of the observability building blocks — metrics, logs, traces, profiling, and alerting and the tradeoffs between them at scale. - Time-series and telemetry data fundamentals: cardinality control, retention and downsampling, query performance, and hot/cold storage tiering. - Live-site reliability craft: SLA/SLO definition, on-call, incident management, and postmortem-driven continuous improvement. - Ability to review and raise the bar on technical artifacts (design docs, code reviews) across a distributed-systems codebase. ## **Preferred Experience** - Applying AI/LLM techniques to triage, anomaly detection, or incident summarization, with clear-eyed judgment about where they help and where they don't. - OpenTelemetry at depth: collector architecture, semantic conventions, instrumentation strategy. - The open-source observability stack in production: Prometheus, Mimir/Thanos/Cortex, Grafana, Loki, Tempo, ClickHouse. - Building observability *as a product* for external developers, not only as internal tooling. - Kubernetes internals and multi-cloud delivery across AWS, GCP, and Azure. - Vendor consolidation or migration programs (commercial APM/SaaS to open-source, or the reverse) with measurable cost and reliability outcomes. - Developer experience or engineering productivity backgrounds - treating internal engineers as customers with a measurable experience. - Prior experience at a developer infrastructure or open-source-based company, where the community is part of the feedback loop. Temporal Technologies is an Equal Opportunity Employer. Temporal Technologies does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status, or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need. We embrace and celebrate differences and diversity. Temporal is committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, its services, programs, and activities. If you need to request a reasonable accommodation, please let your Recruiter know so we can assist.