The short version: gpt-oss runs under either server, but they're built for opposite jobs. vLLM is a datacenter throughput engine: it runs the model in its native MXFP4, batches many requests at once, and is the right backend when gpt-oss is serving a real agent workload. llama.cpp is a portability engine: it runs a GGUF quant on almost any hardware — one consumer GPU, CPU-only, a mixed box — where vLLM won't even load. If you're standing up a production agent backend on an 80GB GPU, pick vLLM. If you're running the model on a laptop, an edge box, or a non-datacenter GPU, pick llama.cpp. Both expose an OpenAI-compatible endpoint, so this is a hardware decision, not a code rewrite.
The one axis that decides it: concurrency vs. portability#
Everything else follows from this. vLLM's signature feature is continuous batching — it keeps many requests in flight and packs the GPU. That's exactly the shape of an agent backend, where multiple loops each fire tool call after tool call. Published benchmarks put gpt-oss-120b around 58 tokens/sec on a single node with MXFP4, and it scales out across nodes.
llama.cpp optimizes the other direction: run anywhere. It takes a GGUF-quantized gpt-oss and runs it on a single card, on CPU, or split across CPU and GPU. Throughput is lower — community runs report roughly 15 tokens/sec generation at low batch with a 4-bit quant — but it will run the model on machines vLLM flatly refuses.
vLLM answers "how many agents can this GPU serve at once?" llama.cpp answers "can I run this model on the hardware I actually have?"
Where vLLM wins#
- Concurrency. Continuous batching means one 80GB GPU serves many simultaneous agent sessions, not one at a time.
- Native format. gpt-oss ships in MXFP4 and vLLM runs it as-is — no re-quantization, no format drift from the weights OpenAI trained.
- Throughput per dollar under load. When the GPU is busy, vLLM extracts far more tokens/sec from it than a single-stream engine.
- Mature path. The vLLM + gpt-oss recipe is well-trodden and documented in the OpenAI Cookbook.
What it means: if the model backs production traffic, vLLM is the default. The only reason not to is that you don't have a datacenter-class GPU to put it on.
Where llama.cpp wins#
- Hardware you already own. A GGUF quant of gpt-oss runs on a single consumer GPU, CPU-only, or a mixed box — no 80GB card required.
- Local dev and offline. Perfect for building the agent on a laptop, or running in an air-gapped environment.
- Grammar-constrained tool calls. llama.cpp can force outputs to match a grammar, which keeps tool-call JSON valid on smaller quants.
- Zero-dependency footprint. A compiled binary and a
.gguffile, no Python inference stack.
What it means: llama.cpp is the portability and homelab choice. When "runs at all on this machine" beats "runs fast under load," it's the answer.
The caveat to test before you trust it#
llama.cpp's flexibility comes with a moving-target risk on gpt-oss specifically: some builds have produced incoherent or off-prompt output on longer prompts — reported around 800+ tokens — with certain GGUF quants (llama.cpp #17016). It gets patched, then a regression reappears elsewhere. The MXFP4 path in vLLM has been steadier because it runs the format the model was trained in.
The safe play: pin a known-good llama.cpp build and a reputable quant (an Unsloth UD-Q4_K_XL is a common pick), then run a smoke test at your real context length before you ship. If answers degrade as prompts grow, that's the failure signature — don't discover it in production.
The decision, in one line each#
- Production agent backend, 80GB GPU, concurrent traffic → vLLM. Native MXFP4, continuous batching, OpenAI-compatible.
- Laptop, edge, homelab, non-datacenter GPU → llama.cpp. GGUF, runs anywhere, pin your build and test long prompts.
- Both, honestly. vLLM for the shared endpoint, llama.cpp for local dev against the same model — one agent codebase, because both speak the OpenAI API.
Neither choice touches your agent loop. Serve gpt-oss under whichever server matches your hardware, point base_url at it, and keep reasoning_effort low on the cheap turns.
