The '60%' attached to vLLM v0.28.0 refers to DSpark time-to-first-token in the release notes and to a different per-user speedup in the paper. Here is the mechanism and how to measure it yourself.
A headline circulating after the vLLM v0.28.0 release reads "Ships Sparse Attention and 60% Faster Speculative Decoding". Three different documents contain a "60" in the same neighbourhood, and they measure different things: a time-to-first-token (TTFT) improvement in the release notes, a per-user generation speedup in the DSpark paper, and a rewording of the first in a news headline. An operator who plans capacity from the headline would be planning from a number whose baseline, metric and hardware are all missing.
The question this article answers: what does the 60% figure actually refer to, how does the mechanism behind it (DSpark confidence-scheduled, or adaptive, verification) work, and how should you measure it on your own traffic?
Background: why fixed-length speculation breaks at high load
Speculative decoding splits generation into a cheap draft step and an expensive verification step in which the target model scores all drafted tokens in one forward pass. Accepted tokens are nearly free; rejected ones are wasted compute. At low concurrency the GPU is under-utilised, so verifying extra draft tokens costs little. At high concurrency the verification batch is already compute-bound, and every low-probability draft token you verify displaces a useful token from another request. A fixed number of speculative tokens is therefore a compromise that is wrong at both ends of the load curve.
The DSpark paper (arXiv:2607.05147, submitted July 6, 2026, 33 authors) states the problem the same way: parallel drafters propose long blocks in one pass but acceptance falls quickly because tokens lack inter-token dependencies, and verifying whole blocks wastes batch capacity on tokens likely to be rejected.
How it works: draft with confidence, verify with a budget
According to the vLLM engineering post of August 14, 2026, the mechanism has four parts.
- Drafting with confidence. DSpark drafts a block of k tokens per pass (
num_speculative_tokens) and a learned confidence head attaches a score to each position. The paper pairs a parallel backbone with a lightweight sequential module to reduce "suffix decay", where later tokens in a block are accepted less often. - Survival probabilities. The scheduler turns confidences into survival probabilities, the running product along each request. Because survival only decreases with position, a global top-B selection over all slots yields a contiguous prefix of each request's draft.
- Choosing B. B is picked to maximise expected tokens per unit of step time: the numerator is one bonus token per sampling request plus the survival of the B best slots, and the denominator comes from a profiled cost table indexed by the step's token count.
- Cost model. At startup the engine times dummy steps across fixed shapes, takes the median of five runs per shape, and builds two lookup tables (verification by token count, drafting by request count), forced monotonic to smooth noise.
Per the post, budget sizing runs on the CPU from a one-step-old confidence buffer, while per-request allocation runs on the GPU with current values, and varlen decode CUDA graphs let one graph serve any mix of 1 to k+1 tokens per request. The post summarises the effect as behaving "like a long fixed block at low concurrency and a short one at high concurrency".
Deep analysis: three different "60"s
| Source | Statement | Metric and baseline | Label |
|---|---|---|---|
| vLLM v0.28.0 release notes (released 26 Aug; the page excerpt I read did not show the year) | "adaptive speculative token budget delivering ~60% better DSpark TTFT" (PR #51725) | TTFT; no baseline, model or hardware stated in the notes I read | Vendor-reported |
| AlphaSignal news item | Headline "60% Faster Speculative Decoding"; body cites a "55 to 65% improvement in end-to-end time-to-first-token on DSpark workloads" | TTFT; no baseline or hardware; cites the release page | Secondary, rewording a vendor claim |
| DSpark paper abstract | "60 to 85 percent" per-user generation speedup | Versus the production baseline MTP-1, at matched throughput, on live DeepSeek-V4 traffic | Authors-reported, not independently reproduced |
These are not the same quantity. TTFT is dominated by queueing and prefill behaviour, while per-user generation speed is a decode-phase property. A 60% TTFT improvement says little about tokens per second, and the headline's "faster speculative decoding" blurs the two. My interpretation: the release-note figure and the paper figure share a number by rounding, not because they describe one experiment. I also could not confirm how PR #51725 (adaptive budget) relates to PR #47808 (confidence-scheduled verification) beyond both appearing under DSpark in the notes, so I do not claim they are the same code path.
The vLLM post's own benchmark is more informative about conditions: DeepSeek-V4-Pro-0813 on 8 x B300 (SM100) with tensor parallel 8, expert parallel and an FP8 KV cache, vLLM main at commit 73b8394, 880 prompts at temperature 1.0 with up to 2048 output tokens, concurrency swept from 1 to 256, against fixed speculation lengths (k at or above 5) and no speculation. Its stated result is that adaptive verification "stays on the edge of the Pareto curve for the whole sweep". The numbers appear only in a figure; the text gives no tokens-per-second values, so I cannot quote any.
Comparison with alternatives
| Approach | Verification length | Strength | Weakness |
|---|---|---|---|
| No speculation | None | Predictable, no draft cost | Leaves decode latency on the table at low load |
| Fixed k (e.g. MTP, Eagle-3, DFlash) | Same for every request and step | Simple; no profiling step | Over-verifies at high concurrency, under-verifies at low |
| Adaptive verification (DSpark, vLLM) | Per-step global budget B from confidence and a cost table | Tracks the load curve in the vendor sweep | Needs a trained confidence head, startup profiling and specific attention backends |
One caveat on comparability: the vLLM AMD speculative-decoding post (August 23) says that in its vLLM path the DSpark confidence head was not active, and that the full proposal is verified. Results from that post are therefore not evidence for adaptive verification.
Practical example: a reproducible sweep
The following server command is copied from the vLLM post. The benchmark loop is my own sketch, untested; confirm flag names with vllm bench serve --help for your version, and substitute your own prompts.
vllm serve deepseek-ai/DeepSeek-V4-Pro-0813 \
--tokenizer-mode deepseek_v4 --trust-remote-code \
--tensor-parallel-size 8 --enable-expert-parallel \
--kv-cache-dtype fp8 --max-model-len 16384 --max-num-seqs 256 \
--max-num-batched-tokens 16384 --gpu-memory-utilization 0.8 \
--compilation-config '{"max_cudagraph_capture_size":4096}' \
--speculative-config '{"method":"dspark","attention_backend":"FLASH_ATTN","num_speculative_tokens":7,"draft_sample_method":"probabilistic","enable_adaptive_verification":true}'
# Untested sketch: repeat per arm (adaptive, fixed k, no speculation)
for c in 1 16 32 64 128 256; do
vllm bench serve --model deepseek-ai/DeepSeek-V4-Pro-0813 \
--dataset-name custom --dataset-path my_prompts.jsonl \
--num-prompts 880 --max-concurrency $c \
--save-result --result-filename "adaptive_c$c.json"
done
Record, per concurrency level: output tokens per second (system throughput), per-user tokens per second, median and p99 TTFT, and p99 inter-token latency. Plot throughput against per-user speed, the interactivity-versus-throughput view the vLLM post uses, and compare arms on that frontier rather than on a single number. Run each point several times; the vLLM post does not report variance.
Limitations and failure modes
- Hardware and backend coupling. Per the post, full varlen decode graphs need
AttentionCGSupport.ALWAYS, which the DeepSeek-V4 backends report on SM100. Elsewhere, adaptive verification is rejected at startup rather than falling back to piecewise graphs. - Unsupported features.
--enforce-eager, LoRA and pipeline parallelism are not supported, and output logprobs are rejected because verification compacts logits after the forward pass. - Checkpoint dependence. The gain requires a drafter with a confidence head; the vLLM benchmark covers one model and one GPU generation.
- Sampling. The reported sweep used temperature 1.0 with probabilistic draft sampling; I found no measurement at greedy settings, where acceptance behaves differently.
Engineering recommendations
- Treat "60% faster" as a pointer to a PR, not a planning number. Ask which metric, which baseline and which concurrency.
- Test adaptive verification only if your traffic spans a wide concurrency range; at a steady, saturated load a well-tuned fixed k may be close.
- Check the v0.28.0 breaking changes before upgrading, including the raised default
max_num_batched_tokens(8192 to 16384) and renamed KV-offload metrics, since both can move your baseline. - Keep a no-speculation arm in every benchmark; some configurations in other vendor posts fell below it.
Conclusion
Confidence-scheduled verification is a sound idea: spend verification compute where survival probability is high and the batch can afford it. The marketing number is not evidence of it. Next step: run the six-point concurrency sweep above on your own prompts with three arms (adaptive, your current fixed k, no speculation) and decide from the throughput-versus-per-user-speed frontier.
Sources
- vLLM v0.28.0 release notes (GitHub)
- Adaptive Verification in vLLM: DSpark confidence-scheduled verification (vLLM blog)
- DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation (arXiv:2607.05147)
- AlphaSignal: vLLM v0.28.0 Ships Sparse Attention and 60% Faster Speculative Decoding
- Exploring Speculative Decoding in vLLM on AMD GPUs (vLLM blog)
Verification note: I read these pages through a fetch tool that summarises them; I did not run vLLM, did not reproduce any benchmark, did not read the full DSpark paper, and the benchmark loop above is untested.
