Ant Group open-sourced LingBot-Video, the world's first mixture-of-experts (MoE) based embodied video foundation model for robotics, achieving a state-of-the-art score of 0.620 on the RBench benchmark. The model is released under Apache 2.0 license on GitHub. This open-source release significantly lowers the barrier for embodied AI research, providing a highly efficient MoE architecture that activates only 3B of 30B total parameters, making it three times more efficient than dense models of similar size. It can accelerate progress in robot action prediction, simulation data generation, and world model development. LingBot-Video innovates in three aspects: architecture (DiT+MoE for capacity-cost balance), data (70K hours of embodied data covering dexterous manipulation, robot locomotion, and egocentric interaction), and training (multi-dimensional reinforcement learning rewards focusing on physical plausibility and task completion). The model uses a Diffusion Transformer (DiT) backbone.
Background
Mixture of Experts (MoE) is an AI architecture that uses multiple specialized sub-networks (experts) and a gating mechanism to activate only a subset for each input, improving efficiency. Diffusion Transformers (DiT) replace traditional U-Net backbones with transformers in diffusion models, enabling better scalability and performance. LingBot-Video combines these technologies for embodied video generation, which learns to produce videos of robots performing tasks, useful for robot learning and simulation. RBench is a benchmark designed to evaluate robot manipulation video generation.