#vLLM

vLLM v0.24.0 was released with 571 commits from 256 contributors, adding support for the MiniMax-M3 model and extensive performance optimizations for DeepSeek-V4, including a FlashInfer sparse index cache and cluster-cooperative topK kernel. This release significantly expands the model support of vLLM and demonstrates its growing role as a high-performance inference engine for cutting-edge LLMs. The optimization for DeepSeek-V4 improves throughput and latency, benefiting large-scale deployment. The release introduces Model Runner V2 (MRv2) supporting quantized models by default, a new streaming parser engine for tool-call parsing, and integration of DeepEP v2 for expert parallelism. Also, vLLM no longer sets CUDAVISIBLEDEVICES internally, replacing it with a deviceids argument.

vLLM v0.25.0 makes Model Runner V2 the default for all dense models, removes the legacy PagedAttention implementation, and achieves performance parity between the Transformers modeling backend and native vLLM. This release is a major architectural overhaul that simplifies the codebase, eliminates technical debt, and broadens model support, benefiting the LLM inference community with improved performance and compatibility. The release includes 558 commits from 232 contributors, adds new models like LLaVA-OneVision-2 and GLM-5, and introduces a new Streaming Parser Engine for tool-call/reasoning parsing.

A benchmark of Gemma 2 9B with FP8 quantization served on a single NVIDIA L4 GPU via vLLM reveals that FP8 increases time-to-first-token (TTFT) by up to 58% for long-context prompts, while reducing end-to-end latency for medium-length generation. This analysis exposes the hidden prefill tax of FP8 quantization on commodity hardware, helping engineers make informed trade-offs between latency, quality, and VRAM when deploying self-hosted LLMs. The unquantized model had a TTFT of 866.93ms for complex long-context prompts, while FP8 spiked to 1372.12ms; however, FP8 reduced average client total time from 12.29s to 11.53s for medium-length sequences and freed VRAM for larger batch sizes.