#SGLang

SGLang v0.5.15 introduces Speculative Decoding V2 with zero-overhead scheduling, achieving +11% end-to-end tokens per second, and IndexShare Multi-Token Prediction (MTP) that reduces draft-step cost by up to 1.9x on long contexts. Additionally, GLM-5.2 with NVFP4 quantization on Blackwell GPUs now serves over 500 tokens per second per user on 8x B300. These optimizations significantly reduce inference latency and cost for production LLM serving, especially on NVIDIA's latest Blackwell architecture. The performance gains in speculative decoding and quantization make high-throughput, low-latency serving more accessible for large models like GLM-5.2, DeepSeek-V4, and others. Key improvements include Spec V2's CUDA-graphable draft-extend with fused metadata ops, IndexShare MTP reusing indexer top-k across steps, TopK V2 fusing top-k with page-table transform, and GEMM shape-specialized JIT routers for Blackwell. The release also enables breakable CUDA Graph by default and adds linear-attention kernels (KDA/GDN) for alternative attention mechanisms.

SGLang v0.5.14 introduces support for multiple new models including GLM-5.2, LiquidAI LFM2.5, and Kimi-K2.7-Code, achieves a 5x throughput boost for DeepSeek-V4 on NVIDIA GB300, and adds two novel MoE load balancing methods (Waterfill and LPLB) for DeepEP. This release significantly improves inference performance for large MoE models like DeepSeek-V4, making it more practical for production use on cutting-edge hardware. The Waterfill and LPLB load balancing methods address a key bottleneck in expert parallelism, potentially reducing latency and increasing throughput for many recent models. The 5x throughput improvement on GB300 is enabled by a combination of optimized kernels, MoE load balancing, and NVFP4 quantization. The Waterfill method dynamically assigns shared-expert work to less-loaded ranks, while LPLB uses linear programming to balance tokens across redundant expert replicas. These are opt-in features (e.g., --ep-dispatch-algorithm=lp).