Lead DevOps Engineer (AI & GPU)

Job description

**This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Lead DevOps Engineer (AI & GPU) based in India (Bangalore, Remote).** As a Lead DevOps Engineer (AI & GPU), you will lead the design, deployment, and operation of the infrastructure that powers production-grade AI services and high-performance inference workloads. You will build robust CI/CD pipelines, optimize GPU-based model serving, and establish reliable delivery processes across cloud, on-premises, edge, and air-gapped environments. Your work will ensure AI applications are deployed securely, operate efficiently, and remain observable throughout their lifecycle. You will oversee platform observability, resource utilization, cost attribution, and governance to deliver scalable, transparent, and economically sustainable AI operations. Collaborating with platform, security, data, and engineering teams, you will integrate automation, continuous verification, and policy enforcement into every stage of delivery. This role offers the opportunity to shape advanced AI infrastructure while solving complex challenges involving performance, security, reliability, and multi-tenant operations. ### Accountabilities: - Lead the design, implementation, and operation of CI/CD and continuous verification (CV) pipelines for AI services, agents, and runtime components, covering automated builds, testing, releases, validation, and rollback. - Implement and manage GitOps-based deployment workflows using tools such as Argo CD, Helm, GitHub Actions, GitLab CI, or equivalent technologies to ensure consistent and repeatable delivery. - Operate and optimize production-scale AI inference and model-serving infrastructure using frameworks such as Ray Serve, vLLM, NVIDIA Triton, or NVIDIA NIM across GPU-enabled environments. - Manage AI asset governance and discovery, including artifact and container registries, model metadata, experiment tracking, upstream model repositories, software supply-chain security, license management, and lineage tracking. - Integrate tools such as Harbor, Artifactory, Nexus, MLflow, Hugging Face, NVIDIA GPU Cloud (NGC), Trivy, and software bill of materials (SBOM) scanning into secure AI delivery workflows. - Design and maintain a comprehensive observability platform encompassing metrics, logs, traces, dashboards, alerts, service-level agreements (SLAs), and AI-specific debugging and evaluation capabilities. - Build and operate OpenTelemetry collection infrastructure, including multi-tenant receivers, exporters, processing pipelines, batching, dynamic filtering, and Kafka-based telemetry delivery. - Implement GPU monitoring and performance visibility through tools such as the NVIDIA Data Center GPU Manager (DCGM) exporter, enabling accurate utilization tracking and infrastructure optimization. - Establish AI FinOps practices by implementing quotas, rate limits, utilization monitoring, and per-tenant cost attribution using tools such as OpenCost, Kubecost, or equivalent platforms. - Ensure that AI workload consumption, telemetry, and runtime behavior are consistently associated with tenant and workload identities to support accountability, cost control, and end-to-end traceability. - Integrate security, quality, integrity, compliance, and cost controls into deployment pipelines, ensuring releases meet defined policies before reaching production. - Connect observability and evaluation signals to deployment decisions, enabling progressive rollouts, automated rollback, and continuous improvements based on operational performance. - Support secure and consistent AI runtime operations across public cloud, private infrastructure, edge deployments, and air-gapped environments. - Collaborate with platform engineering, security, trust, and data teams to establish clear ownership boundaries, service expectations, consumption policies, and operational contracts. - Communicate architectural decisions and technical trade-offs related to AI infrastructure, cost management, runtime reliability, and observability to engineers, architects, and leadership. ## Requirements: - Bachelor's or Master's degree in Computer Science or a related technical discipline. - At least 8 years of professional experience in DevOps, MLOps, platform engineering, observability engineering, or related fields, including hands-on production ownership of AI workloads or high-throughput services. - Strong expertise in CI/CD pipeline design, continuous verification, GitOps practices, automated testing, progressive delivery, and production release and rollback strategies. - Practical experience with GitOps and automation tools such as Argo CD, Helm, GitHub Actions, GitLab CI, or equivalent technologies. - Demonstrated experience operating production-scale AI inference and model-serving systems, including Ray Serve, vLLM, NVIDIA Triton, or NVIDIA NIM, on GPU infrastructure. - Strong understanding of GPU-enabled workloads, inference performance optimization, resource allocation, and the operational requirements of high-throughput AI services. - Extensive knowledge of observability technologies, including Prometheus, Grafana, Loki, Tempo, Jaeger, and OpenTelemetry, covering metrics, logs, distributed tracing, alerting, and dashboard design. - Experience with API gateways and request-routing technologies such as Envoy or equivalent, including support for streaming responses and efficient AI request distribution. - Familiarity with AI model and artifact governance, container registries, software supply-chain security, vulnerability scanning, SBOMs, license compliance, and model lineage. - Experience implementing cost-management and FinOps practices using OpenCost, Kubecost, or equivalent tools, including resource quotas, rate limiting, utilization monitoring, and workload-level cost attribution. - Ability to design and operate infrastructure consistently across cloud, on-premises, edge, and air-gapped environments, with an emphasis on security, reliability, and operational consistency. - Proven experience managing production infrastructure subject to security controls, compliance requirements, or regulatory constraints. - Strong analytical and problem-solving skills, with the ability to troubleshoot complex distributed systems and improve performance, reliability, and cost efficiency. - Excellent communication and collaboration skills, including the ability to explain technical architecture, operational risks, and cost trade-offs to both technical and non-technical stakeholders. - Ability to establish clear service ownership, consumption contracts, and operational expectations across platform, security, trust, and data teams. - Experience with LLM evaluation and debugging tools such as LangSmith or Langfuse, including prompt and response quality measurement, is an advantage. - Familiarity with secure sandboxing and workload isolation technologies such as gVisor, Firecracker, or microVMs is beneficial. - Additional experience with GPU telemetry through DCGM, OpenLineage, AI license management, multi-tenant SLA monitoring, and per-tenant alerting is desirable. - Exposure to regulated industries such as financial services, healthcare, taxation, or risk management is an advantage. ## Benefits: - **Remote work flexibility:** Opportunity to work remotely from India, with Bangalore identified as the role's location. - **Full-time employment:** Join a technical team focused on building and operating advanced AI infrastructure and production-grade systems. - **Technical leadership opportunities:** Influence AI platform architecture, deployment standards, observability strategies, and GPU infrastructure operations. - **Advanced technology exposure:** Work with modern AI inference frameworks, GPU computing, cloud-native infrastructure, GitOps, and distributed observability platforms. - **Professional development:** Expand your expertise in AI infrastructure, MLOps, FinOps, platform engineering, and secure production deployment. - **Cross-functional collaboration:** Partner with engineering, platform, security, and data teams to solve complex infrastructure challenges. - **Meaningful technical impact:** Improve the reliability, scalability, security, transparency, and cost efficiency of production AI workloads. - **Inclusive workplace:** Opportunity to contribute to a professional environment committed to equal employment opportunities. - **Salary and additional benefits:** Compensation details, healthcare coverage, paid leave, bonuses, and other specific benefits have not been disclosed in the available job description and should be confirmed during the hiring process.