vLLM v0.24.0 was released with 571 commits from 256 contributors, adding support for the MiniMax-M3 model and extensive performance optimizations for DeepSeek-V4, including a FlashInfer sparse index cache and cluster-cooperative topK kernel. This release significantly expands the model support of vLLM and demonstrates its growing role as a high-performance inference engine for cutting-edge LLMs. The optimization for DeepSeek-V4 improves throughput and latency, benefiting large-scale deployment. The release introduces Model Runner V2 (MRv2) supporting quantized models by default, a new streaming parser engine for tool-call parsing, and integration of DeepEP v2 for expert parallelism. Also, vLLM no longer sets CUDA_VISIBLE_DEVICES internally, replacing it with a device_ids argument.
Background
vLLM is an open-source inference engine for large language models (LLMs), designed for high throughput and low latency. It features efficient memory management with PagedAttention and supports various models. The optimizations in this release leverage techniques like FlashInfer for sparse attention and MXFP4 for low-precision computation, which are critical for handling large models like DeepSeek-V4 and MiniMax-M3.