vLLM: DeepSeek-V4.1-Flash decode throughput within 6% of DeepSeek's own DeepInfer engine; 954K tokens/s per GPU on a GB300 NVL72
vLLM's Oct 23 blog: on GB200 under DeepSeek's decode setup, decode throughput is within 6% of DeepInfer's published results at batch sizes 8-256; DeepInfer's numbers were read from its charts, not run side by side. On a GB300 NVL72 with P/D disaggregation and Wide-EP, vLLM hits 954K tokens/s per GPU on SemiAnalysis AgentX at P90 196 tokens/s per user, about $38.96 per GPU-hour at off-peak prices.
Why it matters
vLLM says it now matches DeepSeek's own DeepInfer engine on DeepSeek-V4.1-Flash decode.
Sources2
vLLM BlogMediaServing DeepSeek-V4.1-Flash at Scale: vLLM Matches DeepInfer and Scales to GB300 NVL72View original ·
- @vllm_projectXvLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64. DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and extensively optimizing prefill and decode, vLLM now matches DeepInfer's performance curves.View original · 2 links ·
Written by dalao.ai: the title, summary and why it matters are our own words. The original reports are the record; their publishers keep authorship.