#inference

vLLM v0.24.0 was released with 571 commits from 256 contributors, adding support for the MiniMax-M3 model and extensive performance optimizations for DeepSeek-V4, including a FlashInfer sparse index cache and cluster-cooperative topK kernel. This release significantly expands the model support of vLLM and demonstrates its growing role as a high-performance inference engine for cutting-edge LLMs. The optimization for DeepSeek-V4 improves throughput and latency, benefiting large-scale deployment. The release introduces Model Runner V2 (MRv2) supporting quantized models by default, a new streaming parser engine for tool-call parsing, and integration of DeepEP v2 for expert parallelism. Also, vLLM no longer sets CUDAVISIBLEDEVICES internally, replacing it with a deviceids argument.

vLLM v0.25.0 makes Model Runner V2 the default for all dense models, removes the legacy PagedAttention implementation, and achieves performance parity between the Transformers modeling backend and native vLLM. This release is a major architectural overhaul that simplifies the codebase, eliminates technical debt, and broadens model support, benefiting the LLM inference community with improved performance and compatibility. The release includes 558 commits from 232 contributors, adds new models like LLaVA-OneVision-2 and GLM-5, and introduces a new Streaming Parser Engine for tool-call/reasoning parsing.

SGLang v0.5.14 introduces support for multiple new models including GLM-5.2, LiquidAI LFM2.5, and Kimi-K2.7-Code, achieves a 5x throughput boost for DeepSeek-V4 on NVIDIA GB300, and adds two novel MoE load balancing methods (Waterfill and LPLB) for DeepEP. This release significantly improves inference performance for large MoE models like DeepSeek-V4, making it more practical for production use on cutting-edge hardware. The Waterfill and LPLB load balancing methods address a key bottleneck in expert parallelism, potentially reducing latency and increasing throughput for many recent models. The 5x throughput improvement on GB300 is enabled by a combination of optimized kernels, MoE load balancing, and NVFP4 quantization. The Waterfill method dynamically assigns shared-expert work to less-loaded ranks, while LPLB uses linear programming to balance tokens across redundant expert replicas. These are opt-in features (e.g., --ep-dispatch-algorithm=lp).