Choosing a Speculative Decoding Method in vLLM: Eagle-3, DFlash, DSpark and Tuning N

Published speculative-decoding speedups on AMD GPUs range from 2.87x to below baseline depending on model, dataset and proposal length. This article explains the methods and a sweep to tune N yourself.

The same speculative-decoding setting can give 2.87x on one model and a slowdown on another. In the vLLM team's August 2026 measurements on AMD GPUs, a DFlash drafter reached 2.87x output throughput on gemma-4-26B-A4B-it (MATH500, N=7), while EAGLE-3 on Qwen3-8B fell below the non-speculative baseline on MATH500 and below 1.0x at low N on several datasets. A team that copies a configuration from a blog post inherits none of the conditions that produced it.

The question: given the draft methods vLLM now serves (native MTP, EAGLE-3, DFlash, DSpark) and the vLLM Speculators library for training drafters, how do you choose a method and tune the proposal length N for your own model and traffic?

Background: where drafters come from

Speculative decoding needs a drafter. Some models ship one (native MTP heads, or the Gemma 4 "assistant" checkpoint that shares the target's KV cache). Otherwise you train one against the target's hidden states. The vLLM Speculators project exists for that second case: its GitHub description is a library for training speculative-decoding draft models that deploy directly into engines such as vLLM. Its documentation lists four algorithms on the page I read (Eagle-3, P-EAGLE, DFlash and DSpark) and states that they produce output from the same distribution as the target model (lossless). The same page says "all seven algorithms" are lossless while listing four, so it is internally inconsistent about scope; I rely only on the four listed.

How the methods differ

Eagle-3 drafts autoregressively with Llama-style draft layers, conditioned on hidden states from early, middle and late target layers (AMD/vLLM post). Several small forward passes produce N tokens.
DFlash predicts a whole block in one forward pass using block diffusion. Per the Speculators v0.5.0 post (May 28, 2026), queries use non-causal attention within a block, and training samples a fixed number of "anchor" positions so the attention mask stays manageable on long sequences. The DFlash paper (arXiv:2602.06036, ICML 2026) is by Jian Chen, Yesheng Liang and Zhijian Liu; a search summary credits it with up to 6x lossless acceleration on Qwen3-8B, but I did not open the paper and treat that figure as author-reported and unchecked.
DSpark adds a lightweight Markov head to a DFlash-style backbone for inter-token dependence and a confidence head for per-position acceptance (Speculators docs). The AMD post notes the confidence head was not active in its vLLM path.
Native MTP uses heads built into the target model and drafts sequentially, with no separate checkpoint.

Deep analysis: what the AMD measurements do and do not say

The August 23, 2026 vLLM post ran these methods on AMD Instinct MI300X and MI355X (mostly tensor parallel 2 or 4) over GSM8K, MATH500, HumanEval and MBPP. The authors say the figures are the best values within each sweep, not guaranteed results. Selected output-throughput speedups versus no speculation:

Target	Method	Reported speedup	Conditions
gemma-4-26B-A4B-it	DFlash	2.87x / 2.79x	MATH500 / HumanEval, N=7
gemma-4-26B-A4B-it	Gemma 4 MTP	2.74x / 2.62x	GSM8K / MBPP, N=5
gemma-4-26B-A4B-it	EAGLE-3	2.11x to 2.27x	Across the four datasets
gemma-4-31B-it	DFlash	2.34x / 2.05x	MATH500 N=7 / HumanEval N=11
Qwen3-8B	DSpark	1.15x to 1.63x	N=7 best in the tables
Qwen3-8B	EAGLE-3	Below baseline on MATH500	Below 1.0x at low N on several datasets
Kimi-K2.5	EAGLE-3 / DFlash	up to 2.33x / up to 2.68x	N=4 / MATH500 N=7

These are vendor-reported, single-stack measurements. Two lessons follow from the post's own caveats. First, acceptance rate alone does not predict throughput, because draft cost matters; a method with a lower acceptance rate and a cheaper draft can win. Second, the longest proposal is not always fastest: extra positions add drafting and verification cost. My reading of the Qwen3-8B rows is that a small, already-fast target leaves little verification cost to amortise, so the drafter's own cost can erase the gain; the post does not state this explanation.

Comparison: choosing a method

Situation	Reasonable first candidate	Why (interpretation)
Model ships MTP heads	Native MTP, N from 1 upward	No training or extra checkpoint to maintain
Published drafter exists for your target	That drafter (for example the RedHatAI Gemma 4 31B DFlash speculator named in the v0.5.0 post)	Zero training cost; verify acceptance on your data
Your own fine-tuned or private model	Train with Speculators, starting with Eagle-3	Docs call it the most established, with mature vLLM support
Long, reasoning-style outputs, deep blocks	Evaluate DFlash or DSpark	One-pass block drafting; the AMD post found medium-to-deep N often good on math

Practical example: train, serve, sweep N

The Speculators v0.5.0 post describes an online workflow: a vLLM server runs the base model, training prompts are sent to it, hidden states are written to disk or a RAM disk, and the trainer loads and deletes them. DFlash adds flags --speculator-type dflash, --block-size and --max-anchors. Serving DFlash needs vllm>=0.20.0. The first line below is verbatim from the AMD post; the sweep is my own untested sketch.

# From the vLLM AMD post (Gemma 4 26B-A4B DFlash)
--speculative-config '{"method":"dflash","model":"z-lab/gemma-4-26B-A4B-it-DFlash","num_speculative_tokens":15,"attention_backend":"triton_attn"}'

# Untested sketch: sweep N, restart the server for each value
for n in 1 3 7 11 15; do
  vllm serve "$TARGET" --speculative-config \
    "{\"method\":\"dflash\",\"model\":\"$DRAFTER\",\"num_speculative_tokens\":$n}" &
  SERVER_PID=$!
  # wait for readiness, run the same prompt set, save results, then:
  kill $SERVER_PID; wait $SERVER_PID 2>/dev/null
done

For each N, record output throughput against a no-speculation run, mean accepted length, overall acceptance rate, and per-position acceptance rate. The AMD post recommends the last one: if later draft positions no longer contribute, reduce N. For DFlash with a block size of 16, the maximum N is 15; the post suggests testing 3, 7, 11 and 15.

Limitations and failure modes

Dataset mismatch. All four benchmark datasets are math or code; the post notes code workloads often preferred moderate N and math preferred medium to deep N. Chat or retrieval traffic is not covered.
Figure-only evidence. The v0.5.0 post compares a Gemma 4 31B DFlash speculator with Eagle 3 and an FP8 verifier but gives no values or hardware in the text.
Batch size. Benefits depend on batch setup, as the AMD post warns, and the vLLM DSpark work exists precisely because fixed blocks over-verify at high load.
Drift. A drafter trained on one prompt distribution can lose acceptance when traffic shifts; I found no published decay numbers.
Moving support matrix. Docs describe DFlash and DSpark support as improving rapidly, and not every method was tested on every model.

Engineering recommendations

Never adopt a published N. Sweep it on a sample of your production prompts and keep a no-speculation arm.
Prefer per-position acceptance over a single acceptance rate when picking N.
If you must train a drafter, harvest hidden states from real traffic (online mode) so the training distribution matches serving.
Re-run the sweep after each vLLM minor upgrade and each drafter retrain.

Conclusion

Method choice and N interact with model size, dataset and batch size, and the published speedups range from clearly positive to slower than baseline. Next step: take 500 real prompts, run the N sweep above for one candidate drafter plus a baseline, and look at per-position acceptance before committing a configuration.

Sources

Verification note: figures come from vendor posts read through a summarising fetch tool (the AMD post's last ~9,000 characters were not read); I did not run any drafter or benchmark, did not open the DFlash paper body, and the sweep script is untested.