At 4:09 a.m. UTC on Oct. 20, DeepSeek engineer Jiashi Li pushed the first commit to a new GitHub repository. The message read "initial commit." The repository, DeepInfer, holds the inference framework behind DeepSeek's own service, and the deployment notes that came with it included a figure model companies rarely publish: a single cluster serves 155 trillion tokens a day, earning $38 per GPU-hour at off-peak prices.
Three days later the team behind vLLM, the open-source inference engine, posted its own results. Testing on GB200 under DeepSeek's decode configuration, it said its decode throughput was within 6% of DeepInfer's published results across batch sizes 8 to 256, and that its prefill tracked DeepInfer's published curve. The post also gave a second number: running the same model, DeepSeek-V4.1-Flash, on SemiAnalysis's AgentX benchmark on one Nvidia GB300 NVL72 rack, vLLM calculated $38.96 per GPU-hour at DeepSeek's off-peak prices. That figure converts throughput into revenue; it is not a comparison with DeepInfer. A day after that, the team behind SGLang, another open-source engine, ran DeepInfer's own benchmarks on the same hardware as DeepInfer (4 GB300s, 4 B300s and 64 GB300s) and reported up to 1.04 times DeepInfer's prefill throughput, decode steps up to 7.4% shorter, and results within 1% at the remaining points.
A model company released the engine it runs in production, and two open-source projects said they had matched it within a week. The few percentage points matter less than what the episode shows: for open-weight models, the work of making a model run fast is turning into shared property. The gap is moving to the parts DeepSeek kept, its cache storage, load balancing and day-to-day operations, and to the price that ends up on the bill. For most independent developers and small teams, the payoff won't come from running the model themselves. It will come as more, and cheaper, places to rent it.
A no, 18 months earlier
DeepSeek didn't start out planning this. On April 14, 2025, it published a note in its open-infra-index repository explaining why it would not open-source its full internal engine. It gave three reasons: the code was an early vLLM fork more than a year old, heavily customized for its own models; it was tightly coupled to internal infrastructure such as cluster management tools; and "as a small research team focused on developing better models, we lack bandwidth to maintain a large open-source project." Instead it promised to split out reusable pieces as standalone libraries and to sync inference work with open-source projects before each model launch, so the community could have full support on day one.
This release resolves only part of that list. DeepInfer's README says the release provides offline inference, and that three things required for a complete service are not included: a KV cache system, disaggregation of the vision encoder, prefill and decode stages, and load balancing across data-parallel replicas, though the README adds that for disaggregation "most required engine functionality is already implemented." The deployment notes say the KV cache management and load balancing components are "tightly coupled to our internal infrastructure." By the evening of Oct. 25 the repository, public for five days, had received 198 pull requests and merged two.
Why the match came so fast
Part of the explanation is that these engines are close relatives. DeepInfer's README says it "was originally built primarily on vLLM," and its core engine file opens with "Derived from vLLM; modified by DeepSeek." In the other direction, vLLM says it has been integrating kernels from DeepSeek's libraries, including MegaAttn, MegaMoE and Mega-mHC. DeepInfer's notes say its asynchronous scheduler follows existing implementations in SGLang and NanoFlow. DeepSeek itself says its internal kernels are "broadly the same" as the open ones, with end-to-end performance "at most 5% apart."
The hardest pieces, the compute kernels, were already public. What the engines competed on was the engineering around them. In an Oct. 7 post, vLLM said that in the three weeks after the model's release it had improved throughput 1.9 times at low concurrency and 5.3 times under a 150 tokens-per-second-per-user constraint.
The claims of parity deserve a discount. Both projects published their own test results. vLLM measured its decode numbers on GB200 and read DeepInfer's off the charts in its README, which says decode was measured on 64 GPUs. vLLM also says its prefill setup differs, so the gap between the curves should not be read as an engine-to-engine speedup. SGLang says some of its settings, such as balanced expert routing, "exist for the comparison only."
What DeepSeek kept
If engine efficiency is close to a commons, the parts DeepSeek held back are where the advantage sits. Its notes describe roughly 100 PiB of SSD devoted to KV cache, about 14.4 terabytes of logical capacity per online GPU, with cache kept for three days for agent traffic and seven for chat. DeepSeek says a small share of chat cache gets read many times over, and cutting retention to about three days costs chat traffic 0.5% to 1% of its hit rate.
The cache shows up in the price. On DeepSeek's rate card, off-peak input for V4.1-Flash costs $0.003 per million tokens on a cache hit and $0.15 on a miss, a 50-fold difference. For agent apps that resend long contexts again and again, the hit rate moves the bill more than a few points of engine speed.
Scale is the other barrier. DeepSeek recommends decode at 64-way expert parallelism or more and writes that "the smallest efficient deployment unit is 64 GPUs of prefill plus 64 GPUs of decode." vLLM's result came from a 72-GPU rack. Small teams don't rent at that size.
What small teams get is a price
Where small teams feel the change is the API market. At 2:17 p.m. UTC on Oct. 25, OpenRouter listed 34 endpoints for DeepSeek-V4.1-Flash-1017 from 29 providers. DeepSeek's own listing showed its off-peak price: $0.15 per million input tokens and $0.60 per million output. Eight providers charged exactly $0.30 and $1.20, identical to DeepSeek's peak price. Six charged less than DeepSeek for output; the cheapest, Sail Research, listed fp4 precision at $0.15.
Cheaper isn't necessarily equivalent. The endpoints are labeled fp8, fp4 or "unknown," and an fp4 endpoint may not produce the same output as DeepSeek's own service. Cache pricing tells its own story. Among the 34 endpoints, only DeepInfra and Decart charged less than DeepSeek for a cache hit. Many charged $0.006, twice as much, and the most expensive charged $0.08. An engine can be reused. A 100-PiB cache cluster can't.
DeepSeek also released a component called deepseek-recipe, which converts between mainstream API protocols and the model's token sequences and aims to give third-party deployments the same API behavior as the official one. If it works as described, switching providers gets cheaper. When choosing one, quantization, cache pricing and peak-hour rates are better questions than whose engine is faster.
Traps on a single node
Teams that want to run the model themselves hit a different set of problems. Warmctl, which makes a tool for GPU cold starts, published a comparison on its blog by Kirill Bespalov, who tested both engines on one 8×B300 node, "the largest unit most teams rent." He first found vLLM two to four times faster on decode and eight to nine times faster on prefill.
On Oct. 24 he updated the post: the gap most likely came from a configuration issue. In DeepInfer pull request #200, an outside contributor showed that on some cloud virtual machines, GPU-initiated RDMA silently falls back to a CPU proxy. "Nothing is logged at WARN level, and everything still works, only slower," the request says. Bypassing that path on an 8×B200 node made prefill 1.44 to 1.79 times faster, reaching 91% to 104% of the README curves, according to the request. As of Oct. 25 it was still open. Bespalov wrote that the eight-to-nine-times gap should be read "as DeepInfer on one cloud node without that fix, not as the difference between the engines."
He also logged a 930-second cold start, of which loading weights took 96 seconds; the rest went to compilation, kernel warmup, autotuning and CUDA graph capture. Warmctl sells a product that shortens that time, and the 88-second restore figure comes from its own test. But problems like these, driver and network configuration, cold starts, monitoring, are where small companies can find business once engine speed converges. None of it shows up in a benchmark chart.
Where the money went
Open-source engines now have venture capital behind them. The creators and core maintainers of vLLM started Inferact, which says that as of January 2026 it had raised $150 million at an $800 million valuation in a round led by a16z and Lightspeed. The vLLM post says the work "was led by Inferact." RadixArk, the company from the SGLang team, announced a $100 million seed round at a $400 million post-money valuation on May 5, led by Accel. Inferact's founding post put the goal this way: "Today, deploying a frontier model at scale requires a dedicated infrastructure team. Tomorrow, it should be as simple as spinning up a serverless database."
Matching DeepInfer is also the best sales material either company could ask for. Their interests broadly line up with their users', since both want inference to be cheaper, but the numbers they publish also serve their own business.
There are reasons to think DeepInfer won't fade into a reference implementation. It supports batch-invariant inference and deterministic sampling, which matter for offline jobs that need reproducible results. It runs on Huawei's Ascend 950, which neither the vLLM nor the SGLang post lists. And DeepSeek says inference code for its future models will keep landing in DeepInfer. When the next model ships, the fastest engine on day one may again be DeepSeek's own.
For this model, at least, the business figure DeepSeek put in its documentation has become a yardstick others measure against. The $38 is DeepSeek's revenue; the $38.96 is vLLM's estimate on a different workload, and the two can't be compared directly. Still, a model company that publishes that number alongside its engine no longer treats the engine as something to hide.