#MoE

Ant Group open-sourced LingBot-Video, the world's first mixture-of-experts (MoE) based embodied video foundation model for robotics, achieving a state-of-the-art score of 0.620 on the RBench benchmark. The model is released under Apache 2.0 license on GitHub. This open-source release significantly lowers the barrier for embodied AI research, providing a highly efficient MoE architecture that activates only 3B of 30B total parameters, making it three times more efficient than dense models of similar size. It can accelerate progress in robot action prediction, simulation data generation, and world model development. LingBot-Video innovates in three aspects: architecture (DiT+MoE for capacity-cost balance), data (70K hours of embodied data covering dexterous manipulation, robot locomotion, and egocentric interaction), and training (multi-dimensional reinforcement learning rewards focusing on physical plausibility and task completion). The model uses a Diffusion Transformer (DiT) backbone.

Tencent has released and open-sourced the Hunyuan Hy3 preview, a Mixture-of-Experts (MoE) language model with 295 billion total parameters and 21 billion active parameters, supporting a 256K token context length. This release significantly enriches the open-source LLM ecosystem with a large-scale MoE model focused on complex reasoning and agent tasks, potentially accelerating development in AI-powered coding and scientific applications. The model achieves a 54% reduction in first-token latency for products like CodeBuddy, thanks to deep co-optimization of model architecture and inference framework. It targets enhanced performance in math, science, and code generation.

Huawei has open-sourced OpenPangu-2.0-Flash, a Mixture-of-Experts (MoE) large language model with 92 billion total parameters and 6 billion active parameters, supporting a 512K token context length. The release includes model weights, inference code, and training operations. This open-sourcing provides the community with a high-performance, long-context MoE model from a major tech company, potentially lowering the barrier for researchers and developers to experiment with large-scale MoE architectures. It also signals Huawei's growing involvement in the open-source AI ecosystem. The model uses an MoE architecture where only 6B of the 92B parameters are activated per token, enabling efficient inference. It achieves a 512K token context window, and a larger Pro variant (505B total, 18B active) is slated for release in July.

SGLang v0.5.14 introduces support for multiple new models including GLM-5.2, LiquidAI LFM2.5, and Kimi-K2.7-Code, achieves a 5x throughput boost for DeepSeek-V4 on NVIDIA GB300, and adds two novel MoE load balancing methods (Waterfill and LPLB) for DeepEP. This release significantly improves inference performance for large MoE models like DeepSeek-V4, making it more practical for production use on cutting-edge hardware. The Waterfill and LPLB load balancing methods address a key bottleneck in expert parallelism, potentially reducing latency and increasing throughput for many recent models. The 5x throughput improvement on GB300 is enabled by a combination of optimized kernels, MoE load balancing, and NVFP4 quantization. The Waterfill method dynamically assigns shared-expert work to less-loaded ranks, while LPLB uses linear programming to balance tokens across redundant expert replicas. These are opt-in features (e.g., --ep-dispatch-algorithm=lp).