You picked an open model — a Llama, a Qwen, one of NVIDIA's new Nemotron 3 weights. Now you have to run it somewhere that isn't your laptop, and five names keep coming up: Together, Fireworks, Baseten, Modal, DeepInfra. Comparison posts line them up feature-by-feature and drown you. Skip that. The five are not really five choices. They're two, and the fork is billing.
The one decision: do you pay per token, or rent the GPU by the hour? Everything else — who's fastest, who's cheapest, whose console you like — is a tiebreaker inside whichever side of that fork your traffic puts you on. (This is a separate question from which inference engine runs underneath the API — vLLM vs SGLang vs Ollama — a choice you often don't even control on serverless.)
The fork: token meter vs. GPU meter#
Serverless per-token means the provider keeps a fleet of the popular models permanently warm and shared across every customer. You send tokens, you're billed for tokens, and when you send nothing you pay nothing — with no idle GPU on your tab and no cold start you have to manage. This is DeepInfra, Fireworks, and Together's serverless tier.
Dedicated GPU-by-the-hour means a GPU (or several) is yours. You pay for wall-clock time it's provisioned, whether it serves one request or a million. That's the native model for Baseten and Modal, and an option Together and Fireworks also offer.
The reason this is the decision: a dedicated H100 runs roughly $4–6.50/hour, which is about $3,000–$4,700 a month kept warm — a flat cost that does not care how busy the GPU is. Per-token pricing has no floor but no ceiling either. So the whole question reduces to one number: is your GPU busy enough that the hourly rate beats the token bill?
When serverless per-token wins (most founders, most of the time)#
If your traffic is low, spiky, or unproven — you're pre-PMF, prototyping, or running a feature a fraction of users touch — serverless is correct and it isn't close. You pay only for output, there's no idle burn between bursts, and there's no ops. A standard catalog model on serverless is the lowest-total-cost path until you have real, sustained volume.
Within serverless, the tiebreakers:
- DeepInfra is the price floor. Tokens start around $0.06/1M, and a 70B-class model runs about $0.35 in / $0.40 out per million — the cheapest per-token option here, on a no-minimums, pay-per-use plan.
- Fireworks trades a little unit cost for latency. Its custom FireAttention serving stack is tuned for throughput and time-to-first-token, so it's the pick when the model sits in a user-facing path and speed shows up in the UX.
- Together is the broadest menu: serverless plus dedicated endpoints plus rentable GPU clusters plus fine-tuning, under one account. You pay slightly more than DeepInfra on tokens for the option to graduate a workload to dedicated capacity without changing vendors.
When dedicated GPU-by-the-hour wins#
Switch to dedicated only when one of three things is true:
- You're running a fine-tuned or custom model no serverless catalog hosts. Serverless only serves what the provider loads; your weights need a GPU that's yours.
- Sustained volume clears the break-even. Estimate the monthly token bill for the workload; once it would exceed a dedicated GPU's ~$3–4.7k/month at your utilization, the flat rate wins.
- You need predictable tail latency or single-tenant isolation — a hard SLA, or data that can't share a multi-tenant fleet.
Here the tiebreakers are about the idle-cost problem, because a GPU you rent by the hour bleeds money while it waits:
- Baseten defaults to scale-to-zero — drop to zero replicas and pay nothing at rest, at the cost of a cold start (seconds for small models, up to minutes for large containers) on the first request after a scale-down. Billing is per-minute, no monthly minimum; it's the enterprise-leaning choice for hosting your own fine-tunes. (Baseten raised $300M in January 2026, from CapitalG and NVIDIA, and the cold-start engineering is where that shows.)
- Modal bills per second and lets you deploy arbitrary Python — any container, any serving logic, not a fixed model catalog. It also scales to zero, with faster cold starts (~2–5s small, ~15–30s for 7B+). Reach for it when you need custom pre/post-processing or a pipeline, not just an inference endpoint.
The tell that this is now a pricing decision, not a capability one#
All five providers appear on the launch roster for NVIDIA's Nemotron 3 open models — the same weights, offered every way at once. Together and DeepInfra and Fireworks and Baseten serve the Nano tier; Modal, Baseten and DeepInfra serve Super and Ultra; even Ollama Cloud is on the Ultra list. When the identical model is a click away on every platform, the model stops being the differentiator. The bill is.
The rule#
Start serverless per-token — DeepInfra if unit cost dominates, Fireworks if latency does, Together if you want room to grow without switching vendors. Move a specific workload to dedicated (Baseten's scale-to-zero, or Modal's per-second custom serving) the day it hits one of the three triggers: a custom fine-tune, sustained volume past break-even, or a latency/isolation guarantee. Don't provision a GPU for traffic you don't have yet — the flat monthly cost is a bet on utilization, and most early products lose that bet.
