#DeepSeek

DeepSeek and Peking University released DSpark, a speculative decoding framework that accelerates LLM inference by 60% to 85% without compromising output quality. The corresponding models are already available on Hugging Face. This open research approach contrasts with the secretive practices of major US AI labs, highlighting DeepSeek's commitment to transparency and innovation. It enables faster, cheaper LLM inference for a wide range of applications. DSpark introduces semi-autoregressive candidate generation and confidence-scheduled verification to dynamically optimize speculation length and acceptance rate. The framework has been deployed in DeepSeek-V4-Flash and V4-Pro preview, with the full DeepSpec codebase open-sourced on GitHub.

Chinese AI company DeepSeek has begun developing its own AI chip focused on inference, aiming to reduce dependence on NVIDIA and Huawei chips. The project started about a year ago and is still in early stages, with DeepSeek actively recruiting chip design engineers and contacting foundries and storage companies. This move could reshape the AI hardware landscape in China and reduce DeepSeek's vulnerability to US export controls, which currently restrict access to advanced NVIDIA chips. If successful, it may also intensify competition with Huawei's Ascend series and other domestic chipmakers. The chip is designed specifically for inference, the phase where trained models generate responses, rather than training. DeepSeek previously relied on NVIDIA H800 and Huawei Ascend chips, and founder Liang Wenfeng acknowledged in a rare 2024 interview that chip restrictions are a challenge for the company.

SGLang v0.5.14 introduces support for multiple new models including GLM-5.2, LiquidAI LFM2.5, and Kimi-K2.7-Code, achieves a 5x throughput boost for DeepSeek-V4 on NVIDIA GB300, and adds two novel MoE load balancing methods (Waterfill and LPLB) for DeepEP. This release significantly improves inference performance for large MoE models like DeepSeek-V4, making it more practical for production use on cutting-edge hardware. The Waterfill and LPLB load balancing methods address a key bottleneck in expert parallelism, potentially reducing latency and increasing throughput for many recent models. The 5x throughput improvement on GB300 is enabled by a combination of optimized kernels, MoE load balancing, and NVFP4 quantization. The Waterfill method dynamically assigns shared-expert work to less-loaded ranks, while LPLB uses linear programming to balance tokens across redundant expert replicas. These are opt-in features (e.g., --ep-dispatch-algorithm=lp).