Every write-up of speculative decoding leads with the same number: some model now runs N percent faster. It's the wrong thing to stare at. The speedup was never the scarce resource. The draft model was — and until this week, if you self-hosted an open model for your agents, producing a good one for your model was somewhere between a research project and a thing you didn't do.
In late June 2026, DeepSeek open-sourced DeepSpec under the MIT license, and the interesting part isn't a faster DeepSeek. It's that DeepSpec ships the training pipeline for the piece everyone else keeps as an internal artifact.
What speculative decoding actually spends#
The mechanism is a small trick with a sharp dependency. A small, fast draft model proposes a short run of tokens; the large target model verifies that whole run in a single forward pass and accepts the longest prefix it agrees with. When the draft guesses well, you get several tokens for roughly the cost of one target step. When it guesses badly, you paid for a proposal you threw away.
So the entire economics reduce to one quantity: the acceptance rate — how often the target ratifies the draft's guesses.
Acceptance rate is not a property of the algorithm. It's a property of the pair — this draft, matched to that target, on this kind of text.
That's the detail the speedup headlines bury. You cannot buy a generic draft model and expect a generic win, because a draft trained to shadow one target on one traffic distribution is the thing doing the work. The algorithm is the easy, publishable part. The matched draft is the part that lives or dies on data and training you have to actually run — which is why, for most teams serving an open model, speculative decoding stayed a slide, not a deployment.
What DeepSpec hands you#
DeepSpec is a unified codebase for exactly that training, collecting three algorithms — DSpark (confidence-scheduled speculative decoding with semi-autoregressive generation), DFlash, and Eagle3 — behind one pipeline. You choose the approach by swapping a config, not by rewriting a trainer.
The two decisions that signal who this is for:
- The default targets are not DeepSeek's models. Out of the box, DeepSpec trains drafts for Qwen3 at 4B, 8B, and 14B, plus Gemma-4-12B-it — the open models people actually self-host. HuggingFace checkpoints are provided, but the repo's point is the recipe, not the artifacts.
- The pipeline is a rerunnable three-stage recipe. Data preparation downloads prompts and regenerates target answers to build a target cache; training fits the draft against that cache; evaluation measures acceptance rate on benchmark tasks. Each stage is a config away from your own setup.
Point stage one at your prompts instead of a public set, and the draft you get is tuned to how your agents really talk. That is the move a generic checkpoint can't make for you.
Why this is an agent story, not just an inference story#
Speculative decoding helps some workloads far more than others, and agents sit on the good side of that line. Agent output is unusually predictable: tool calls are templated JSON, responses repeat schema fields, code carries heavy structural regularity, and the same boilerplate recurs turn after turn. Predictable text is high-acceptance-rate text — the draft's guesses land more often — so the per-request latency win tends to be larger for an agent emitting its fifth structured tool call than for open-ended prose.
That flips the usual read. Speculative decoding is often framed as a chat-latency optimization; for a self-hosted agent stack it's closer to a throughput-and-tail-latency lever aimed straight at your most repetitive, most frequent generations. And because DeepSpec lets you train the draft on your own traffic distribution, you're optimizing acceptance for the exact shapes your agents produce, not for a benchmark's.
The honest caveat, and the number that matters#
DeepSpec cites an accompanying paper (arXiv:2607.05147) for its speedup claims, and the eye-catching percentages circulating in secondary coverage come from there. Treat those as reported until you've read the tables — or, better, until you've measured acceptance rate on your own workload, because that's the number that actually sets your latency. A published benchmark tells you the pair can work; your own eval tells you whether it works on your traffic.
The shift worth internalizing is smaller than a percentage and more durable: a model shipping with native multi-token prediction is not automatically the speed ceiling, and a well-matched draft you trained yourself is now a config file rather than a paper you admire. For anyone running an open model in production for agents, the barrier to speculative decoding just moved from "we'd need to build the draft" to "we'd need to run the pipeline." Those are very different sentences.
