vLLM 0.26.0 and SGLang 0.5.16 both shipped on July 25, 2026 — the two dominant open inference engines releasing the same day, again. Last time they did this, the whole story was the scheduler "sync stall": which engine kept the GPU fed while the CPU planned the next batch. This round, that story didn't get a sequel. Neither release headlines a scheduler-overlap or async-scheduling change. The sync-stall race is, for practical purposes, settled — so the real decision moved to three other axes: speculative decoding, prefix caching, and which new models you can serve on day one. Here's the honest read for a founder choosing where to serve.
The non-event that's actually the headline#
If you came in expecting a "sync stall, round two," here's the finding: there isn't one. vLLM 0.26.0's scheduler-adjacent changes are incremental (grammar-compile failures no longer crash the engine; better prefix-cache reporting). SGLang 0.5.16's only scheduler note is a micro-tweak — moving a barrier to right after each batch launch. The CPU-overhead gap that defined the 0.25 vs 0.5.15 fight closed quietly, and both teams spent this cycle elsewhere. That's good news: for a single-node deployment, you can now largely stop worrying about scheduler overlap and choose on the feature that matches your traffic.
SGLang 0.5.16: throughput on a fixed model#
SGLang aimed this release at getting more tokens out of a model you've already picked.
- UnifiedRadixTree is now the default prefix cache. This is the line to notice. Agent and RAG traffic resends the same long prefix — system prompt, tool schema, retrieved chunks — every turn; a radix-tree cache reuses the already-computed KV instead of recomputing it. Making it the default means you get that reuse without tuning. If your prefixes overlap heavily, this is the single most impactful change in either changelog.
- DSpark, confidence-driven speculative decoding, is the throughput headline — the release reports ~383 tokens/sec at an accept length around 5 on a DeepSeek-V4-Pro configuration. (Treat the exact decimal as reported by the release notes, not an independent benchmark.)
- Expert-parallel MoE work — EPLB dispatch improvements, DeepEP, elastic EP scaling — for teams running large mixture-of-experts models across GPUs.
vLLM 0.26.0: breadth and memory#
vLLM aimed this release at serving more models, in less memory.
- Day-one support for a new large multimodal MoE family (Inkling), full stack — if being able to serve the newest models the week they drop is your constraint, vLLM remains the safer default.
- Flexible per-KV-cache-group attention backends, so hybrid models (mixing sliding-window and full attention) get the right backend per layer group.
- Tiered KV offloading to CPU and object storage — a way to hold more context than fits in raw GPU memory, which pairs well with the single-GPU serving math for a quantized MoE like Qwen3.6-35B-A3B.
- More quantization paths — INT4 MoE emulation, online NVFP4-per-token MoE, MXFP4 via FlashInfer.
Read the breaking changes before you upgrade#
Neither is a drop-in bump. vLLM 0.26 removed the TeleChat, Persimmon, and Fuyu model implementations. SGLang 0.5.16 dropped QServe and FBGEMM FP8, now requires FlashInfer for NVFP4, renamed several launch flags (--fp4-gemm-backend cutlass is gone), and turns chunked input-logprob on by default. Pin your exact version, read the notes for your specific model and quantization, and test in staging — a boring pinned version beats a surprise OOM in front of users.
The decision#
- Serve on SGLang if you're squeezing maximum throughput out of a fixed model with repeated prefixes — agents, RAG, chat. Its default radix prefix cache and speculative decoding are built for exactly that shape.
- Serve on vLLM if you need the newest models supported day one, want tiered KV offloading to stretch GPU memory, or lean on its broader quantization matrix.
- On a single node, they're close. With the sync-stall gap closed, the deciding factor is which named feature above maps to your workload — not a global "which is faster." If you're still choosing your first engine, the vLLM vs SGLang vs Ollama breakdown starts one level up. Then benchmark both on your own prompts; the cost-per-completed-task number is the one that tells the truth.
