If you read one line: SGLang 0.5.16 (July 25, 2026) added DSpark, a speculative-decoding scheme that stops making you pick a fixed draft length. It drafts in blocks and lets each verification window size itself from the draft's confidence. Turn it on with three things — --speculative-algorithm DSPARK, SGLANG_RAGGED_VERIFY_MODE=compact, and --speculative-dspark-block-size — and expect the big numbers (383.7 tok/s on DeepSeek-V4-Pro) only at low batch size.

Every speculative-decoding scheme is one bet: a small, cheap draft model guesses the next few tokens, the big target model verifies them all in a single forward pass, and every guess it accepts is a token you got for the price of drafting instead of the price of a full decode step. The bet pays because decoding one token at a time on a big model is memory-bandwidth-bound — the GPU is waiting on weights, not busy with math — so you have spare compute to spend verifying speculative tokens for free.

The hard part has always been the same question: how many tokens do you draft before you verify? EAGLE and Medusa make you answer it once, up front, with a fixed number. DSpark's whole idea is that this is the wrong question.

What DSpark actually changes#

Set a fixed draft length and you've committed to one number for a response that isn't uniform. Some spans — boilerplate, the closing of a JSON object, a predictable sentence — the draft model nails ten tokens deep. Other spans — a novel identifier, a hard reasoning step — it whiffs on token two. A fixed length is too timid on the easy spans (you leave free tokens on the table) and wasteful on the hard ones (you draft and verify tokens that get rejected, burning the compute you were trying to save).

DSpark makes the window adaptive. Per the LMSYS write-up, it:

That's the "confidence-driven, variable-length verification" phrase in one sentence: the algorithm reads how certain the draft is and spends verification budget where it'll actually be accepted. This is a different axis from the SGLang spec-decode v2 default work and from DeepSeek's DeepSpec draft-model approach — it's about the shape of verification, not just a faster draft.

Turn it on: the three knobs#

DSpark is a launch-server flag, not a code change. You add it to the same sglang.launch_server command you already use:

python -m sglang.launch_server \
  --model-path deepseek-ai/DeepSeek-V4-Pro \
  --tp 8 \
  --speculative-algorithm DSPARK \
  --speculative-dspark-block-size 8

with one environment variable set for the verification path:

export SGLANG_RAGGED_VERIFY_MODE=compact

Three things are doing the work:

  1. --speculative-algorithm DSPARK selects the algorithm, the same slot where you'd otherwise put EAGLE3.
  2. SGLANG_RAGGED_VERIFY_MODE=compact picks the compact ragged-verification path — "ragged" because the verify windows are now different lengths, so the kernel has to handle variable-length batches instead of a neat rectangle.
  3. --speculative-dspark-block-size is your main dial. It bounds how far the semi-autoregressive drafting reaches before a verify pass; the confidence logic adapts inside that bound.

These are the parameters named in the v0.5.16 release notes and PR #30261. Defaults and per-model draft configuration move between releases, so read the speculative-decoding docs for the exact draft-model path and current defaults for your checkpoint before you ship. In 0.5.16 the work references DeepSeek-V4, Qwen3, and GLM-5.2 — confirm your exact model is on the list, because the draft path is model-specific and DSpark on an unsupported target will either refuse to launch or fall back.

The number, and the asterisk on it#

LMSYS reported 383.7 tokens/sec at an average accept length of ~5 on DeepSeek-V4-Pro, TP8 on a single B300, at batch size 1. Read every clause of that sentence:

Push batch size up and the picture inverts. As you fill the GPU with concurrent requests, decode stops being memory-bound and starts being compute-bound — and now the extra forward-pass work of drafting and verifying rejected tokens is competing with real requests instead of using slack. This is the universal speculative-decoding tradeoff, not a DSpark flaw: it's a latency optimization that quietly costs throughput once the GPU is busy.

Speculative decoding is a trade of spare compute for lower latency. DSpark spends that compute more intelligently — but it can't create spare compute that a full batch has already claimed.

When DSpark is the right call#

When to skip it#

The one thing to do before you trust any of this#

Run it against your model and your concurrency, not the press-release configuration. Spin up the server twice — once with your current decoding, once with DSpark — and measure tokens/sec and p50/p99 latency at the batch sizes you actually see, following the discipline in how to benchmark LLM inference. A single tokens-per-second number from someone else's batch-size-1 B300 run tells you the technique's ceiling. Whether you clear it depends on your acceptance rate, your traffic shape, and how full your GPU already is — and the only way to know is to run the two servers side by side and read the graph.