If you read one line: the fight over whether to self-host Kimi K3 keeps getting argued on the $3/$15 sticker price. That's the wrong number. For an agent, the number that decides it is $0.30 — the cache-hit input rate — and it moves the break-even away from self-hosting, not toward it.
Moonshot published Kimi K3's full open weights and technical report on July 27, 2026: 2.8 trillion total parameters, ~104 billion active per token (16 of 896 experts), ~1.4TB of MXFP4 checkpoint, a 1-million-token context window (Kimi Team technical report). It is a genuinely frontier model — #1 on the Arena.ai Frontend Code Arena (~1,679, ahead of Claude Fable 5) and 93.4% on SWE-bench Verified on Vals AI's independent harness, third behind GPT-5.6 Sol and Fable 5. The moment weights land, the reflex fires: now I run it myself and stop paying per token. Three good posts on this site have already done the hardware math and reached the same verdict — rent, don't own. This piece is about the variable all of them left out.
The sticker price is not what an agent pays#
Kimi K3's API has two input prices, not one (kie.ai, OpenRouter):
- $3.00 / M input on a cache miss — the first time the model sees those tokens.
- $0.30 / M input on a cache hit — a 90% discount, applied automatically, with no separate cache-write fee.
- $15.00 / M output, flat across the full 1M-token window.
Moonshot credits its Mooncake split-inference architecture: caching is automatic, so you never mint a cache ID or set a TTL. Send the same 400K-token repo, the same long system prompt, the same tool schema on turn after turn, and only the new tokens each turn pay the $3 rate. Everything the model has already seen is billed at thirty cents.
That is exactly the shape of an agent. A coding agent re-reads the same codebase on every step. A support agent re-sends the same policy docs. A research agent carries a growing transcript. These workloads are mostly re-sent context — which means their effective input rate collapses toward the $0.30 floor, not the $3 ceiling. Model a repo-reading agent honestly, and the real per-million input you pay can be a small fraction of the sticker.
Every rent-vs-own spreadsheet that plugs in $3/M input is describing a workload that doesn't cache. Your agent isn't that workload. Price the cache hit, and the API gets an order of magnitude cheaper before you've touched a GPU.
The cluster gets no discount#
Here's the asymmetry that flips the decision. A self-hosted deployment also caches — vLLM shipped day-0 K3 support, and both vLLM and SGLang do automatic prefix caching that reuses the KV cache for a shared prefix. But on your own cluster, a cache hit saves compute, not dollars.
You rented the GPUs by the hour. K3 doesn't fit on a single node — an 8×H200 box (~1.13TB VRAM) is smaller than the 1.4TB weights alone, so the realistic floor is 16×H200, an 8×B300 / GB300 NVL72, or 16×B200, and full-precision serving points at 64 accelerators (Northflank). That cluster runs on the order of $26k–$90k+ a month, and the bill is identical whether your prefix-cache hit rate is 5% or 95%. A great hit rate means you can pack more concurrent users onto the same silicon — real value, but it shows up as throughput headroom, never as a smaller invoice.
So put the two side by side. On the API, caching is a price cut you see on the statement. On the cluster, caching is capacity you've already paid for. The same technical phenomenon — reusing a KV prefix — lands as money on one side and as nothing-visible on the other.
Which way the break-even moves#
The three earlier decision pieces put the crossover at roughly a billion tokens a month at high, steady utilization — the point where a fixed cluster beats variable API spend. That was computed against $3/M input. Re-price the input at an agent's blended rate — a large slice at $0.30 — and the API bill for the same traffic drops, which means you have to push even more sustained volume before the cluster wins.
Caching doesn't nudge the line toward self-hosting. It shoves it further away. The cheaper the API gets per useful token, the more saturation your $26k–$90k cluster needs to justify itself — and a solo founder almost never keeps 16+ datacenter GPUs saturated.
There's a second-order trap worth naming: output isn't cacheable. The $15/M output rate is fixed no matter how good your caching is, and agents that think out loud (K3 ships an always-on reasoning mode) generate a lot of it. If you're hunting for the real lever on your K3 bill, it's the output column and your reasoning_effort setting, not the decision to buy GPUs. Turn reasoning down on high-volume, low-stakes calls; that saves more, faster, than any self-hosting plan.
The honest verdict#
Self-hosting Kimi K3 is the right call for exactly the reasons it was before caching entered the picture — and none of them is price:
- Data residency / air-gap. If customer data legally can't leave your perimeter, inference in your VPC is the only compliant path. Cost is irrelevant.
- Fine-tuning. The API gives you the base model; the weights let you change it. A K3 tuned on private data is a moat the endpoint can't sell you.
- Sustained saturation. If you genuinely keep the cluster near 100% around the clock, the fixed cost amortizes — but measure a real month of token volume first, and price the API side with the cache discount before you believe you're here.
For everyone else — which is almost every team of one — the move is unchanged and now even clearer. Point your agent at the K3 API, let Mooncake cache your context for you, watch your effective input rate fall toward thirty cents, and spend the GPU-provisioning weekend shipping instead. The largest open-weight model ever made you a gift. The gift isn't a checkpoint you house in a rack — it's a 90% discount you get for free by not moving.
