Engineering Manager, LLM Inference & Deployment at Scale

Job description

NVIDIA is seeking an engineering manager to lead engineering activities related to productizing Deep Learning models. Academic and commercial groups around the world are using GPUs to redefine Artificial Intelligence and data analytics, and to power data centers. Join the team building software which will be used by the entire world. Interact with the scientific community to implement and improve the latest algorithms. Ability to work in a multifaceted, product-centric environment is required and excellent interpersonal skills are also a requirement. If you have a good understanding for deep learning and a strong algorithmic background, with exposure to large scale LLM/VLM deployment, inference optimization, and leadership experience, then this role may be a great one for you! In this role you will lead and mentor forward-thinking engineers and will own related activities and interactions with teams across NVIDIA. You will be working with key internal partners on priority alignment across relevant teams for roadmap development of highly optimized novel and state-of-the-art numerical, analytics, and deep learning algorithms and associated R&D duties. If the idea of pushing the boundaries of state-of-the-art research and development excites you, and are interested in getting exposure to the entire DL SW stack, come join the team that build the GPU-accelerated DL platform used by the entire world. What you’ll be doing: Plan, schedule, mentor, and lead the execution of projects and activities of the team. Including creating, optimizing, and deploying inference DL workloads. Collaborate with internal customers to align priorities across business units Coordinate projects across different geographic locations Grow and develop a world-class team Travel to conferences, other sites, or visit customers occasionally What we need to see: Minimum requirement of BSc or equivalent experience 8+ overall years related of overall experience, including 3 years of management/leadership experience Experience leading multiple software engineering projects Strong experience with Large Language Models (LLMs) and Large Visual-Language Models (VLMs) Excellent programming, debugging, performance analysis, and test design skills Great communication Ways to stand out from the crowd: Experience with inference of DL models Experience doing performance analysis and tuning Exposure to inference platforms such as TensorRT-LLM, vLLM, and SGLang Project management tools (e.g. JIRA, Microsoft Project) Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 224,000 USD - 356,500 USD for Level 3, and 272,000 USD - 431,250 USD for Level 4. You will also be eligible for equity and benefits . Applications for this job will be accepted at least until July 11, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes. NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.