#NVFP4

SGLang v0.5.15 introduces Speculative Decoding V2 with zero-overhead scheduling, achieving +11% end-to-end tokens per second, and IndexShare Multi-Token Prediction (MTP) that reduces draft-step cost by up to 1.9x on long contexts. Additionally, GLM-5.2 with NVFP4 quantization on Blackwell GPUs now serves over 500 tokens per second per user on 8x B300. These optimizations significantly reduce inference latency and cost for production LLM serving, especially on NVIDIA's latest Blackwell architecture. The performance gains in speculative decoding and quantization make high-throughput, low-latency serving more accessible for large models like GLM-5.2, DeepSeek-V4, and others. Key improvements include Spec V2's CUDA-graphable draft-extend with fused metadata ops, IndexShare MTP reusing indexer top-k across steps, TopK V2 fusing top-k with page-table transform, and GEMM shape-specialized JIT routers for Blackwell. The release also enables breakable CUDA Graph by default and adds linear-attention kernels (KDA/GDN) for alternative attention mechanisms.

NVIDIA has released Qwen3.6-27B-NVFP4, a 27-billion parameter language model quantized to 4-bit floating point using the custom NVFP4 format. This enables efficient inference on compatible NVIDIA hardware, particularly Blackwell GPUs. This release is significant for local LLM deployment, as it offers a strong 27B model with reduced memory footprint and bandwidth requirements. It demonstrates NVIDIA's push to enable high-quality inference on consumer-grade hardware through advanced quantization. The model is based on Qwen3.6 and uses NVFP4, a 4-bit floating-point format introduced with NVIDIA Blackwell architecture. NVFP4 employs a two-level scaling strategy with a fine-grained E4M3 exponent and a secondary FP32 scalar to maintain accuracy at ultra-low precision.