Research Scientist - World Modeling

Job description

About the Institute of Foundation Models We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy. **As part of our team, you’ll have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development.** You will participate in the development of groundbreaking AI solutions that have the potential to reshape entire industries. Strategic and innovative problem-solving skills will be instrumental in establishing MBZUAI as a global hub for high-performance computing in deep learning, driving impactful discoveries that inspire the next generation of AI pioneers. **The Role** We are the **AllWorld Team** under the **Institute of Foundation Model (IFM)** at **MBZUAI**. At AllWorld, we are pioneering the development of the **PAN (Physical, Agentic, and Networked) world models**—the next-generation foundation models to unlock machine intelligence beyond lingual. Our mission is to tackle the fundamental challenges of world modeling and establish a **new paradigm for next-generation machine reasoning**. We are looking for passionate individuals who share our vision and are eager to push the boundaries of AI together. ### Key Responsibilities - Develop the foundational world model to accurately simulate the physical world. - Collaborate with engineering and data teams to tackle key challenges in training the world model on large-scale clusters. - Develop metrics and evaluation benchmarks to better assess model performance. - Design and implement a scalable and efficient data annotation pipeline to ensure high-quality labeled data for training and evaluation. - Optimize inference efficiency to enable real-time interaction. ### Areas of Focus - **Scalable Training Systems**: Develop and optimize infrastructure for training multimodal LLMs and video diffusion models at massive scale. - **Efficient Data Pipelines**: Build scalable video data pipelines and annotation frameworks to support high-quality training data. - **Inference Optimization**: Enhance inference efficiency through optimization and distillation techniques to enable real-time interaction. - **Visual Tokenization**: Develop methods for discretizing visual features into tokens for improved model representation. - **Quantitative Evaluation**: Establish rigorous benchmarks to assess physical accuracy, controllability, and intelligence. - **Scaling Laws for Video Pretraining**: Investigate scaling law principles to guide efficient video pre-training strategies. ### Academic Qualifications - MSc or PhD in Machine Learning or Computer Science, or equivalent industry experience. ### Professional Experience - Experience in large-scale model training (LLMs or Diffusion Models) on large clusters. - Hands-on experience with state-of-the-art video generative models (e.g., Sora, Veo2, MovieGen, CogVideoX, etc.). - Experiences in building and optimizing large-scale video data pipelines. - Experience in accelerating diffusion model inference for improved efficiency. - Exceptional problem-solving and troubleshooting skills to tackle complex technical challenges. - Strong systems and engineering expertise in deep learning frameworks such as PyTorch. - Strong communication and collaboration skills for effective cross-functional teamwork. - Ability to navigate ambiguity and drive projects in rapidly evolving research areas. - Research contributions to top-tier conferences or journals (e.g., ICML, ICLR, NeurIPS, ACL, CVPR, COLM, etc.), with published work in relevant domains.