ML Infrastructure Engineer

Job description

### About the Role This is a hands-on infrastructure engineering role at an early-stage enterprise AI company building a context and data governance layer for AI agents in highly regulated industries. You will own the inference and model-serving infrastructure end to end, ensuring AI agents run reliably, accurately, and at scale in production environments where performance is non-negotiable. ### What You'll Do - Design, build, and operate inference and model-serving infrastructure from development through production deployment. - Scale systems to support AI agents running reliably under increasing concurrency and production load. - Identify and resolve infrastructure bottlenecks in close collaboration with ML and platform engineering teams. - Optimize systems for latency, throughput, and reliability at scale. ### What We're Looking For - 5 or more years building and operating machine learning inference systems, model-serving platforms, or ML infrastructure in production environments. - Hands-on experience designing and scaling inference serving infrastructure using tools such as TensorFlow Serving, TorchServe, Triton, KServe, or equivalent custom systems. - Strong systems engineering fundamentals with expertise in distributed systems, containerization, and orchestration (Docker, Kubernetes). - Demonstrated ability to optimize production ML systems for latency, throughput, and reliability under high concurrency. - Experience with cloud infrastructure platforms such as AWS, GCP, or Azure for deploying and managing ML workloads. - Proficiency with monitoring, observability, and debugging tools such as Prometheus, Grafana, ELK, or distributed tracing frameworks. - Proficiency in at least one systems programming or backend language: Python, Go, Rust, C++, or Java. - Experience with knowledge graphs, semantic search, or graph databases (e.g., Neo4j, Amazon Neptune) is a plus. - Familiarity with agentic AI systems, autonomous agents, or multi-step reasoning pipelines is a plus. - Experience with enterprise data infrastructure, data pipelines, or data integration platforms is a plus. ### Location This role is on-site in San Mateo, California. Visa sponsorship is not available.