If you read one line: gpt-oss-120b is one model, but the same open weights generate tokens roughly 3× faster on wafer-scale silicon than on a fast GPU cloud — and because every provider speaks the same OpenAI-compatible API, switching is a base-URL change, not a rewrite. So don't agonize over the model. Decide by who is waiting: if a human or an agent's next step is blocked on the tokens, buy speed (Cerebras, Groq); if nothing is waiting, buy the cheapest tokens (a GPU cloud). Everything below is how to make that call in numbers.
The one fact that reframes the decision#
Providers publish wildly different speeds for the identical model. In 2026 Artificial Analysis measurements of gpt-oss-120b output speed have looked roughly like this — a shape, not a fixed scoreboard, because the numbers move week to week and by reasoning-effort setting:
| Provider | Output speed (tokens/sec) | Silicon |
|---|---|---|
| Cerebras | ~1,700–1,800 (claims up to ~3,000) | Wafer-scale (WSE) |
| SambaNova | ~700 | RDU dataflow |
| Fireworks | ~600 | GPU (vLLM-class) |
| Together | ~560 | GPU |
| Groq | ~476 | LPU |
The gap between the top and a typical GPU endpoint is about 3×, and every one of these is the same weights answering the same prompt. The only thing that changed is the silicon underneath. (Why the custom chips win — SRAM bandwidth versus the GPU's HBM wall — is the whole story in Groq vs Cerebras vs SambaNova: the race for faster-than-GPU inference; this piece is the buyer's-side sequel to that architecture explainer.)
Because the speed is a provider property and switching is nearly free, the model is not the decision. The provider curve is.
Switching is one base URL#
Every provider above serves gpt-oss-120b behind an OpenAI-compatible endpoint, so "try a different provider" is a config change you can A/B behind one code path:
from openai import OpenAI
# Cerebras — buy speed
client = OpenAI(
base_url="https://api.cerebras.ai/v1",
api_key="CEREBRAS_KEY",
)
# ...or Fireworks — buy cheap tokens. Same code, one base URL changed.
client = OpenAI(
base_url="https://api.fireworks.ai/inference/v1",
api_key="FIREWORKS_KEY",
)
resp = client.chat.completions.create(
model="gpt-oss-120b",
messages=[{"role": "user", "content": "Summarize this ticket in one line."}],
reasoning_effort="low", # the second lever — see below
)That portability is the point: you are not marrying a vendor, you are routing a workload. So route it by the number the workload actually feels.
The rule: pick by who is waiting#
There are only two states, and they want opposite providers.
Latency-bound — a human or an agent is blocked on the output. Chat, coding autocomplete, and tool-calling agent loops all live here. The metric the user feels is time-per-output-token (TPOT) — how fast text streams out — not price. A tool-calling agent makes many sequential model calls per task, so every extra second of generation compounds across the loop. Here, buy speed silicon (Cerebras, Groq). We break down why perceived speed is TPOT, not TTFT, in inference latency: TTFT vs TPOT, and why a fast interactive loop changes agent UX in Cerebras at 750 tokens/sec makes interactive agents feel different.
Throughput-bound — nothing is waiting. Batch summarization, extraction, RAG indexing, and offline eval runs live here. No human sees the latency; you just need the corpus processed by morning. The only felt metric is cost per token, so buy the cheapest capable endpoint (Fireworks, Together, Baseten on a GPU stack) and feed it large batches — the peak single-stream speed of wafer-scale silicon is wasted on a job that doesn't care when each token lands. How to actually measure the three numbers that matter here is in how to measure real LLM cost: tokens, TTFT, throughput.
Most teams end up running both: fast silicon on the interactive path, cheap GPUs on the batch path — same model, two providers, chosen by who is waiting.
The hidden second lever: reasoning effort#
gpt-oss-120b ships a reasoning-effort dial (low / medium / high). It is usually discussed as a quality knob, but it is also a speed and cost knob, and the two interact in a way that changes the provider math.
Higher effort makes the model spend more output tokens thinking before it answers — often several times more. Output tokens are exactly what a slow provider is slow at. So on a latency-bound path, raising reasoning effort on a slow provider costs you twice: more tokens generated, and each token generated slower. A prompt that feels instant at low effort on Cerebras can feel sluggish at high effort on a GPU endpoint — same model, same question. The takeaway: choosing reasoning effort and choosing provider speed are one decision, not two. (The effort dial itself, and how it differs from an explicit thinking budget, is in reasoning effort vs thinking budget.)
Do the arithmetic, per workload#
Skip the vendor loyalty and compute two numbers for your traffic:
- Wait = (output tokens per request) ÷ (provider output speed). This is what a latency-bound user feels. At
higheffort a request might spend 2,000 output tokens: ~1.1s on Cerebras at 1,800 t/s, ~4.2s on Groq at 476 t/s, and multiple seconds more on a GPU endpoint. - Cost = (total tokens per month) × (price per token). This is what a throughput-bound job feels. Speed silicon usually charges a premium per token; a GPU cloud usually undercuts it. For batch work, the cheaper token wins outright.
Whichever number your workload actually experiences is the one to minimize. That's the entire decision — and because the switch is one base URL, you can prove it with an A/B instead of an argument.
The short version#
gpt-oss-120bis one model; provider choice swings output speed ~3×, and switching is a base-URL change.- Latency-bound (human/agent waiting) → buy output speed: Cerebras or Groq.
- Throughput-bound (batch, indexing, evals) → buy cheap tokens: Fireworks / Together / Baseten on GPUs.
- Reasoning effort is a speed lever too: high effort punishes slow providers twice. Tune it with the provider, not separately.
- Verify current numbers — the ranking moves — and prove your pick with an A/B, since the code doesn't change.
