Engineer Aleksandr Samarin has published an explainer of about 12,500 words on his personal site, dated 11 October, that sorts the competing versions of speculative decoding by a single question: how does the small guesser produce its guesses? The technique lets a cheap helper model propose the next several fragments of an answer, which the large model then checks all at once instead of writing them one by one. The framework is Samarin’s own, and it has not been peer reviewed.
Three terms carry the rest of this piece. The helper is a “drafter”. Checking its guesses is “verification”. “Acceptance length” is how many guessed fragments the large model keeps per round, on average, so a higher number means more words for each trip through the big model.
Samarin’s first fork is whether the drafter guesses one fragment at a time or all of them together. EAGLE-3 and multi-token prediction both guess in sequence. EAGLE-3 is a small separate network that reads the big model’s internal state at several depths and then writes guesses one after another. Multi-token prediction is built into the big model during training, so the guesser ships with it. Samarin says DeepSeek-V4 serves with a one-fragment version, and that Qwen3.5 includes the layers natively.
DFlash and DSpark sit on the other branch. DFlash fills in a whole block of upcoming fragments in one pass, which removes the wait for guesses to chain together. The cost is that each position guesses without seeing its neighbors. If the model is torn between “New York” and “Los Angeles”, the block can stitch the two together, and the check rejects it. DSpark, which Samarin credits to DeepSeek, keeps the one-pass block and adds a light step so each guess leans on the one before it.
The fork matters because accuracy is not the whole story. Samarin cites figures from LMSYS in which EAGLE-3 and DFlash on Qwen3-4B both averaged 4.2 kept fragments per round. The speedups were 2.1 times and 3.3 times. The guesses were equally good. The difference was how many times the guesser had to run. He also cites a vLLM benchmark on AMD chips where EAGLE-3 on Qwen3-8B ran at 0.44 to 0.88 of plain speed, meaning slower than not guessing at all.
That last result bears on our 9 October article, in which engineer Joe Barrow argued that the technique only wins while the chip has idle capacity. Samarin’s framework agrees with Barrow, and it is more specific. By his arithmetic, an 8-billion-parameter model on an Nvidia H100 reads its stored numbers in about 4.8 milliseconds, and the chip can check roughly 295 fragments in that time before the math itself becomes the limit. Below that point, extra guesses are nearly free. He notes that serving software fills the same gap by batching users together, and “batching improves throughput, not the latency of each request.”
Where Samarin goes beyond Barrow is in quantifying what happens when the gap fills. In his model (which assumes the guesser is right 80 percent of the time and costs 5 percent of a normal step), one user benefits from a large guess tree, meaning many alternative continuations checked at once. At 128 simultaneous requests the best tree shrinks to a chain of two fragments, and “a 64-node tree there makes decoding almost four times slower than not speculating at all.” Those figures illustrate the shape of the curve and do not measure a real chip. On DSpark, he writes that “under light load every request drafts long; under heavy load verification becomes compute-bound,” so only confident guesses get budget.
The two pieces answer different questions. Barrow explains why the speedup exists and where it ends. Samarin treats load as a design constraint and shows how each family of drafters copes with it. Neither post is a vendor benchmark, and Samarin’s own numbers come from his model and from other groups’ tests.
Before comparing variants, a team should find out how many requests its servers handle at once at the busiest hour. A product serving a few users at a time can pay for elaborate trees. A crowded cluster is likely better off with short chains, or with the setting off.
Aleksandr Samarin, “Speculative Decoding: The Art of Guessing Well”, astralord.github.io, published 11 October 2026.
