Senior DevOps Engineer, Cloud Simulation Infrastructure

Job description

We are seeking a Senior DevOps / Cloud Simulation Infrastructure Engineer to own the complete end-to-end cloud execution pipeline for SimReady assets! This role is critical to our product strategy, enabling us to transition from local, workstation-driven validation to high-scale, automated cloud validation on NVIDIA Cloud Functions (NVCF). You will be responsible for deploying a robust, multi-GPU pipeline that supports structural validation, AI-driven runtime behavioral testing, and automated asset remediation. What you'll be doing: Deployment: Deploy full Isaac Sim runtimes within GPU-aware NVCF containers. Manage container packaging, GPU initialization, and runtime utilities for physics, sensor, and rendering validation. Deploy Structural Validation: Deploy services to validate USD structure and compliance without runtime overhead. Deploy Runtime Validation: Architect scalable execution layers to conduct runtime behavior-based testing (e.g., drop/grasp tests). Deploy rule-based systems or AI based systems for automated pass/fail grading. Deploy Automated Remediation: Develop an AI-based pipeline that intercepts failures, triggers automated asset fixes, and re-validates results to ensure quality standards. Cloud Infrastructure Ownership: Scale execution from single-workstation validation to massive, multi-GPU cloud environments. Optimize for performance, addressing function-to-function networking, gRPC bottlenecks, and in-cluster proxy behavior. Artifact & Evidence Pipeline: Automate the generation of verification videos, thumbnails, feature-level reports, and validation metadata. Ensure all assets are traceable and linked to quality gates. Observability & CI/CD: Establish robust CI/CD, cluster verification, and monitoring pipelines. Implement logging, metrics, and tracing to ensure services are observable, debuggable, and production-ready. Operational Reliability: Implement atomic update semantics and safe failure handling to ensure validation processes never corrupt the primary asset library. What we need to see: BS or MS degree in Computer Science, Computer Engineering, or related field (or equivalent experience). 8+ years of professional experience working on DevOps and/or cloud simulation. Extensive experience in production-grade DevOps, SRE, or Infrastructure Engineering, with a focus on GPU-backed cloud services. Proven expertise in container orchestration (Kubernetes/Docker) and CI/CD pipeline development. Experience with automated testing frameworks, preferably involving AI/ML inference, computer vision, or rule-based validation. Proficiency in Python and systems scripting for test orchestration and pipeline automation. Strong ability to design and maintain distributed job lifecycle services (submit/poll/fetch/cancel) and handle asynchronous failure states. Ability to diagnose and solve distributed network bottlenecks, including gRPC and function-to-function communication. Ways to stand out from the crowd: Direct experience deploying services on NVCF (NVIDIA Cloud Functions) or DGX Cloud. Deep familiarity with Isaac Sim, Omniverse, USD, or Sensor RTX workflows. Background in robotics simulation, physical AI, or large-scale content creation pipelines. Experience building "self-healing" or automated remediation workflows. Experience with cluster verification frameworks, stress testing, and deployment validation at scale. We value different paths to technical excellence and welcome candidates who bring strong judgment, curiosity, and a collaborative approach. Come build the future of autonomous vehicle simulation with us! Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD. You will also be eligible for equity and benefits . Applications for this job will be accepted at least until July 21, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes. NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.