The one-line version: on July 25, 2026, both major open-source inference engines cut releases again — vLLM 0.26.0 and SGLang 0.5.16 — and for the second cycle running they aimed their biggest work at the same target. Two weeks ago that target was the scheduler sync stall. This time it moved down a layer: where your KV cache lives when it stops fitting in VRAM. vLLM's answer is to add tiers below the GPU. SGLang's is to make what's on the GPU smaller and more shareable.

If you self-host open models, this is the layer that decides your cost per concurrent user. Here's what each engine shipped, and how to tell which one fits your bottleneck.

Why memory, and why now#

The thing that kills a self-hosted deployment is rarely raw token speed. It's the KV cache — the per-request scratch the model keeps for every token it has already seen. Serve a long system prompt to a hundred users at once and the cache, not the weights, is what fills your GPU. Once it overflows HBM you have three bad options: cap concurrency, recompute context you already had, or evict and re-fetch. Both engines spent this cycle attacking that overflow — from opposite ends.

vLLM 0.26: push the cache down a tier#

vLLM 0.26.0 (411 commits from 212 contributors) matured its KV offloading into something that looks like a real memory hierarchy. The release notes describe an object-store secondary tier with workload identity, DP-replica-aware tiering, per-tier event handling (BlockStored events), and encoder-cache connectors that can offload to CPU. The idea: cold cache blocks spill to a slower, larger tier instead of being dropped, so you can keep more concurrent context alive than your HBM physically holds.

Alongside it, vLLM made attention itself more granular — you can now select a different attention backend per KV-cache group, and sliding-window support became an explicit backend capability rather than a special case. There's also an fp32 lm_head option via a new head_dtype param for generation models, and DeepSeek-V4 kernel work the notes claim at 1.5–2x on a fused_topk_bias path.

The vLLM bet: your problem is many distinct long sessions that individually blow past VRAM. Give them a hierarchy to live in.

SGLang 0.5.16: shrink and share what's on the GPU#

SGLang 0.5.16 (574 PRs from 169 contributors) went the other way — it made the on-GPU cache smaller and more reusable. The headline is that UnifiedRadixTree is now the default cache for sliding-window, Mamba, and DSA model paths. SGLang's whole identity is RadixAttention — a radix tree that lets many requests share the cache for a common prefix instead of each paying for it — and unifying it as the default extends that reuse to more exotic model families.

The memory numbers are the story. The notes claim a GLM-5.2 DSA cache-layer split that cuts per-rank KV memory ~74% (0.77 → 0.20 GB/rank), and a ReplaySSM Ring Spec-Verify change that shrinks speculative-decoding scratch from 11.5 GB to 1.8 GB per GPU — 6.4x smaller. On the speed side, a new DSpark speculative-decoding method is cited at 383.7 tok/s (accept length ~5) on DeepSeek-V4-Pro at TP8 on B300, and a KDA decode kernel at 29.6 µs vs 36.8 µs for Triton.

The SGLang bet: your problem is many users sharing a long prefix. Don't spill it — dedupe it and shrink it so it never overflows.

vLLM adds floors beneath the GPU. SGLang makes the room on the GPU bigger. Both are answers to "the KV cache doesn't fit" — and which one is right depends entirely on the shape of your traffic.

The number caveat, up front#

Every tok/s and GB figure above is the projects' own release-note claim, on their chosen model and hardware — an Inkling-class MoE here, a DeepSeek-V4-Pro on B300 there. Best cases, in other words. A throughput number is only real once you've reproduced it on your model, your GPUs, and your batch shape, which is the entire argument of how to benchmark LLM inference. Use these to decide what to test, never to size a cluster.

How to choose#

Three questions, in order:

  1. Does the engine support your model? Check the model-support section of the exact release you'll run. Both cycles added large MoE and multimodal stacks (the notes reference families like Inkling and DeepSeek-V4), but coverage differs release to release. An engine 20% faster on a model it doesn't support is useless to you.
  2. What's the shape of your context? Many users on one shared prefix → start with SGLang's radix reuse. Many distinct long sessions past VRAM → start with vLLM's tiered offload.
  3. What breaks if you bump? SGLang 0.5.16 makes UnifiedRadixTree the default (a behavior change for SWA/Mamba/DSA), removes --fp4-gemm-backend cutlass, and renames two flags. vLLM 0.26.0 drops TeleChat, Persimmon, and Fuyu. Read the breaking-changes section before you move a production pin.

The convergence is the real signal. When both engines independently decide the frontier is the memory hierarchy, that's the layer to instrument in your own stack this quarter — because it's the one now deciding your bill.