AI Researcher — Inference Optimization

Job description

ROLE OVERVIEW We are seeking an AI Researcher with deep experience in inference optimization to design, evaluate, and deploy high-performance inference systems for large-scale machine learning models. You will work at the intersection of model architecture, systems engineering, and hardware-aware optimization, improving latency, throughput, and cost efficiency across real-world production environments. KEY RESPONSIBILITIES - Research and develop techniques to optimize inference performance for large neural networks. - Improve latency, throughput, memory efficiency, and cost per inference. - Design and evaluate model-level optimizations (quantization, pruning, KV-cache optimization, architecture-aware simplifications). - Implement systems-level optimizations (dynamic batching, kernel fusion, multi-GPU inference, prefill vs decode optimization). - Benchmark inference workloads across hardware accelerators. - Collaborate with engineering teams to deploy optimized inference pipelines. - Translate research insights into production-ready improvements. REQUIRED QUALIFICATIONS - Strong background in machine learning, deep learning, or AI systems. - Hands-on experience optimizing inference for large-scale models. - Proficiency in Python and modern ML frameworks (e.g., PyTorch). - Experience with inference tooling (e.g., Triton, TensorRT, vLLM, ONNX Runtime). - Ability to design experiments and communicate results clearly. PREFERRED / NICE-TO-HAVE QUALIFICATIONS - Experience deploying production inference systems at scale. - Familiarity with distributed and multi-GPU inference. - Experience contributing to open-source ML or inference frameworks. - Authorship or co-authorship of peer-reviewed research papers in machine learning, systems, or related fields. - Experience working close to hardware (CUDA, ROCm, profiling tools). WHAT SUCCESS LOOKS LIKE - Measurable gains in latency, throughput, and cost efficiency. - Optimized inference systems running reliably in production. - Research ideas successfully translated into deployable systems. - Clear benchmarks and documentation that inform product decisions. RELEVANT RESEARCH AREAS (BONUS) - Long-context inference optimization - Speculative decoding - KV-cache compression and paging - Efficient decoding strategies - Hardware-aware inference design