If you serve a model in production, you are probably renting inference from a managed host — Baseten, Together, Fireworks — and paying by the token or the GPU-minute. At some point the bill gets big enough that someone asks the obvious question: should we just rent the GPUs ourselves and run this in-house?

Here is the short answer, up front, because it is the part most people get backwards. A managed inference host rents you the exact same NVIDIA H100 a neocloud rents bare — and charges roughly 2–3× per hour for it. As of August 2026, a dedicated H100 on Baseten runs about $0.108 a minute, or roughly $6.50 an hour; the identical card rents from RunPod or Lambda for about $2–$3 an hour. So the instinct is: cut out the middleman, pocket the difference. And that instinct is often wrong, because the break-even is not a price — it is a duty cycle. Below a certain utilization, the managed host is genuinely cheaper despite the higher hourly rate. Above it, owning GPUs wins on price — but only if you ignore the ops you just signed up for.

This piece does the arithmetic, then tells you the three conditions that actually have to be true before you move. It is the operational sequel to today's Wire read on why inference became its own $13B category — that piece is why the managed layer exists; this one is when to leave it.

The premium is not a markup — it's three products#

Start by naming what the 2–3× actually buys, because "they're overcharging" is the wrong model. A managed host's rate bundles three things you would otherwise build yourself:

None of that is free when you self-host. It just moves from a line item to a headcount.

The break-even is a duty cycle, not a dollar figure#

Now the math, with the two real rates. Say a bare H100 on a neocloud costs $2.50/hr and the managed equivalent costs $6.50/hr. A bare GPU you rent runs — and bills — continuously: 24 × 30 = 720 hours a month, or ~$1,800/month, fixed, no matter how busy it is.

To spend that same $1,800 on the managed meter at $6.50/hr, you would have to run 277 hours a month — about 9 hours a day, or ~38% of the month.

That 38% is the crossover, and it flips the naive intuition:

Below ~38% GPU duty cycle, the managed host is cheaper than owning one bare GPU — because scale-to-zero means you pay for the 38% you use, while your own card bills for the 62% it sits idle. Above ~38%, the bare GPU is cheaper, because its fixed cost is now spread across enough real work to beat the per-minute rate.

Plug in your own numbers — the rates move, and reserved or committed-use pricing pushes the bare rate down further (dropping the crossover), while spot capacity drops it further still if your workload tolerates interruption. But the shape holds: low or spiky utilization favors the managed meter; steady, high utilization favors the metal. This is the same logic behind serverless GPU vs dedicated when per-second billing wins, applied one layer up — to the managed host itself.

Why the real crossover sits above the price line#

Here is the trap. The 38% figure is a price crossover. It assumes the two options are otherwise identical. They are not — because above that line you are no longer buying GPU-hours, you are buying an operations team's worth of work.

When you move in-house you inherit, at minimum: autoscaling and your own scale-to-zero logic, cold-start latency on a cold GPU, batching and throughput tuning, driver/CUDA upgrade cycles, multi-region failover, observability, and a pager someone carries. That is engineering-months to build and a permanent tax to run. So the honest break-even is not "am I past 38% utilization" — it is "does the money I'd save past 38% exceed the fully-loaded cost of the ops I'm taking on, plus the risk of getting it wrong in production?"

For a solo founder or a small team, that bar is high. The classic mistake is the one covered in rent-a-GPU vs LLM API for the solo founder: running the price math, seeing a saving, and forgetting that the saving has to pay a salary before it pays you. The broader self-hosting vs API cost picture is the same story at the API layer.

The move-off checklist: three conditions, all of them#

Leave the managed host when all three are true — not any one:

  1. Sustained high utilization. Your load keeps GPUs busy well past the crossover duty cycle, steadily, most days. If you're scaling to zero half the day, you have already lost the price argument. Confirm it against a real GPU rental price map at your actual duty cycle, not a peak-hour snapshot.
  2. Ops capacity you can afford. You have — or can hire — the engineering to own serving, and the on-call to keep it up, without starving the product. If inference going down at 2am means nobody answers, you are not ready.
  3. Steady, predictable traffic. Your load doesn't swing so hard that you'd need to rebuild the host's elastic autoscaling to survive launch days. Spiky consumer traffic is exactly what the managed layer is best at absorbing.

Miss any one and the premium is still doing its job. A useful middle path before the full jump: some teams stay on a managed host but move to dedicated (reserved) capacity on it — trading scale-to-zero for a lower committed rate — which is a smaller step than owning the metal outright. The Together vs Fireworks vs Baseten managed comparison is where to price that, and RunPod vs Modal vs Baseten covers the serverless tier below it.

What this means for your roadmap#

Don't move for the hourly rate. The rate is a trap that shows you a saving and hides a team. Move when utilization is high enough, and steady enough, and you can staff the operations — and until then, treat the managed premium as what it is: the cheapest way to rent capability you can't yet build. When the day comes that all three conditions are true, CoreWeave vs Lambda vs Nebius is where you'll go shopping for the metal. Most teams get there later than they think — and the ones who move too early spend their scarce engineering not on the product, but on rebuilding, badly, the exact thing they were already paying someone to run well.