Software Engineer - GPU Fabric Observability

Job description

ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to to ship AI products. THE ROLE Baseten is building its own GPU infrastructure for large-scale inference. As we move into large scale, high-density NVIDIA systems, the hardest failures are intermittent, cross-layer, and difficult to prove: RoCE congestion, InfiniBand stalls, ECN/DCQCN mis-tuning, bad optics, RNIC issues, host kernel stalls, GPU driver problems, and workload symptoms that look like network problems, but are not. We are hiring a Software Engineer to build a first-class observability and root-cause analysis system for GPU fabrics. This is a hard distributed systems problem, not a dashboarding problem. The system will collect high-volume signals from switches, hosts, active probes, and inference services; reduce and correlate them in real time; understand topology and service ownership; and produce actionable diagnosis while an incident is still unfolding. This role sits at the boundary between networking and inference software. RDMA data paths, GPUDirect transfers, prefill/decode disaggregation, KV cache movement, request routing, and workload backpressure can all create fabric symptoms or hide real fabric failures. The goal is to tell an operator, quickly and with evidence, whether an incident is caused by the fabric, host, NIC, GPU, RDMA path, scheduler, or serving layer — and what to do next. EXAMPLE INITIATIVES - Real-time telemetry engine — Build the ingestion, reduction, storage, and query path for high-cardinality fabric, host, GPU, and workload telemetry. - Service-aware fabric diagnosis — Build collectors, probes, and topology-aware correlation to detect latency, drops, stalls, congestion, bad paths, and degradation. - Software-aware RDMA diagnosis — Tie network behavior to RDMA operations, GPUDirect paths, KV cache transfers, prefill/decode disaggregation, and request latency. RESPONSIBILITIES - Own Baseten’s GPU fabric observability and root-cause analysis architecture. - Build telemetry pipelines across switches, NICs, hosts, GPUs, Kubernetes, and inference services. - Model topology, flow paths, service ownership, and failure domains. - Separate true fabric faults from host, NIC, GPU, kernel, driver, RDMA, scheduler, and workload failures. - Create clear operator workflows for triage, remediation, and post-incident learning. REQUIREMENTS - Staff-level or senior staff-level experience building production infrastructure software. - Strong distributed systems background, especially streaming systems, telemetry pipelines, diagnostics, or control-plane software. - Experience building systems that process high-volume, high-cardinality, noisy operational data. - Understanding of networking fundamentals and high-performance networks - Ability to work with low-level infrastructure signals and build practical correlation, anomaly detection, or root-cause analysis systems. BENEFITS - Competitive compensation, including meaningful equity. - 100% coverage of medical, dental, and vision insurance for employee and dependents - Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!) - Paid parental leave - Fertility and family-building stipend through Carrot - Company-facilitated 401(k) - Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities. Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you. At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).