Member of Technical Staff (GPU Performance Engineer)

Job description

We are seeking an experienced GPU Performance Engineer with a strong background in Python and large-scale model training. In this role, you will design and implement improvements to our training infrastructure and directly contribute to technical decisions that optimize performance of our models. You will also work on post-training processes, including reinforcement learning and fine-tuning. Furthermore, you will contribute to improving the efficiency and scalability of our model serving infrastructure. Ideal Experience - Strong engineering skills with fluency in Python and PyTorch (or other frameworks). - Proven experience implementing and training large deep learning models. - Experience writing and debugging low-level GPU code (CUDA, C++). - Experience scaling up GPU jobs using large-scale compute clusters (e.g., Slurm or Kubernetes). - Demonstrated ability to analyze and optimize the performance of GPU-accelerated workloads, including profiling, identifying bottlenecks, and implementing performance tuning techniques. Reka's Mission Reka's mission is to build useful multimodal artificial intelligence and use it to empower organizations and businesses. We are a globally distributed foundation model startup, headquartered in the San Francisco Bay Area, California. Embracing a remote-first approach, our team brings together top talent from around the world. Our founding team, along with many of our team members, has contributed to numerous breakthroughs in AI over the past decade. Why Reka? - An Elite Team: Collaborate with top-tier engineers, researchers, and operators from renowned organizations like Google DeepMind, Facebook AI Research (FAIR), and successful startups, driving innovation in AI technology. - Cutting-edge Infrastructure: Train state-of-the-art models leveraging the latest software and hardware, expanding the frontier of innovation in AI infrastructure development. - Inclusive and Open Culture: Thrive in an open and inclusive work environment that values diverse perspectives and fosters creativity. - Generous Benefits: Enjoy five weeks of paid leave to recharge, comprehensive healthcare benefits (including vision and dental), and additional perks that support your well-being. - Visa Support: We provide visa assistance, including H1B and OPT transfers, for US employees to ensure a smooth transition and support your career with us.