SGLang v0.5.14 introduces support for multiple new models including GLM-5.2, LiquidAI LFM2.5, and Kimi-K2.7-Code, achieves a 5x throughput boost for DeepSeek-V4 on NVIDIA GB300, and adds two novel MoE load balancing methods (Waterfill and LPLB) for DeepEP. This release significantly improves inference performance for large MoE models like DeepSeek-V4, making it more practical for production use on cutting-edge hardware. The Waterfill and LPLB load balancing methods address a key bottleneck in expert parallelism, potentially reducing latency and increasing throughput for many recent models. The 5x throughput improvement on GB300 is enabled by a combination of optimized kernels, MoE load balancing, and NVFP4 quantization. The Waterfill method dynamically assigns shared-expert work to less-loaded ranks, while LPLB uses linear programming to balance tokens across redundant expert replicas. These are opt-in features (e.g., --ep-dispatch-algorithm=lp).
Background
SGLang is an open-source inference engine designed for large language models, particularly Mixture-of-Experts (MoE) architectures. MoE models use multiple 'experts' that are activated per token, requiring efficient communication and load balancing across GPUs. DeepEP is a communication library for expert parallelism, and load imbalance can waste compute. Waterfill and LPLB are runtime dispatch-time methods that redistribute work without changing model weights.