vLLM 0.26.0 shipped on July 25 — 411 commits from 212 contributors — and buried in the changelog are three knobs that change how a self-hosted stack behaves under real load. If you rent your inference, skip this. If you run vLLM yourself, here's what to turn and what it buys.
1. head_dtype: keep the lm_head in fp32 while the body stays quantized#
Quantizing a model to fit your GPUs is standard, but the last layer — the lm_head that projects hidden states into vocabulary logits — is where low precision hurts most: small logit errors flip which token gets sampled. head_dtype lets you set that final projection's precision independently, so the body can run quantized while the head stays fp32. 0.26.0 extends this to LoRA paths and adds a ROCm optimization, so it's no longer a narrow special case.
Reach for it when sampling looks unstable or when structured / constrained decoding is producing off-by-one token choices — the head is cheap enough that full precision there is usually worth it. The rest of the model doesn't need to know.
2. A different attention backend per KV-cache group#
Until now, vLLM picked one attention backend for the whole model. But modern architectures mix attention types — some layers full-context, others sliding-window — and one backend is rarely optimal for both. 0.26.0 lets you select a different attention backend per KV-cache group, and makes sliding-window an explicit backend capability rather than an implicit assumption.
The practical payoff: on a model that interleaves attention patterns, each group can use the kernel that serves it best instead of a lowest-common-denominator choice that leaves throughput on the table. The caveat is honesty about config surface — this is a knob you benchmark, not one you set blind. Measure tokens-per-second on your own model before trusting the change, using the caveats in how to benchmark LLM inference.
3. KV offload grew an object-store tier#
The KV cache is what makes autoregressive serving fast — and what runs out first under long context or high concurrency. When it's full, vLLM's old choice was to evict blocks and recompute them later. 0.26.0 matures the offloading path with an object-store secondary tier (with workload identity) and DP-replica-aware tiering, so cache blocks can spill to object storage and be pulled back on demand instead of thrown away.
The trade is explicit: you pay object-store latency on a cache miss in exchange for not recomputing. For a workload that thrashes local KV memory — big contexts, bursty concurrency — that's usually the better side of the trade, and it's the same "spill, don't recompute" logic that governs where a long-running agent's context should live. For a small, steady workload that fits in local memory, leave it off.
The model list changed — read it before you upgrade#
Two footnotes that aren't footnotes if they touch you:
- New: full support for the Inkling model family (piecewise CUDA graph, speculative decoding, LoRA, NVFP4 quantization), plus DeepSeek-V4 kernel work — a specialized routing kernel and a 1.5–2× faster
fused_topk_bias— that lands as a straight speedup if you serve V4. - Removed: TeleChat, Persimmon, and Fuyu were dropped. If you serve any of them, pin your current version and plan a migration before you bump — an upgrade that deletes your model is a bad way to find out.
The move: if you run DeepSeek-V4 or want the offload and attention work, upgrade and benchmark the three knobs against your own traffic — not a smoke test. If your stack leans on TeleChat, Persimmon, or Fuyu, stay put until you've migrated. Same-week releases earn production the same way every release does: on your numbers, not the changelog's.
Related: we broke down the last inference-engine face-off in vLLM 0.25 vs SGLang 0.5.15, and the engine-choice question in vLLM vs SGLang vs Ollama.
