The one-line version: Embeddings got so cheap that a rented GPU almost never beats the API on raw cost. The cheapest APIs — OpenAI text-embedding-3-small, Voyage-4-lite, Jina v3 — sit around $0.02 per million tokens as listed in mid-2026, and the break-even is a single division: your GPU's fixed monthly cost divided by that per-token price. A ~$0.80/hr L40S run around the clock is about $580/month, so it only undercuts a $0.02/M API somewhere past ~29 billion tokens a month — roughly five times the entire text of English Wikipedia. Against a premium $0.13/M API the crossover falls to ~4.5B tokens; if you already own the GPU it collapses to ~2B. Below your crossover, use the API and move on. Above it — or if you need sub-20ms latency, data residency, or freedom from rate limits — self-host.
This is the sibling to what an AI agent actually costs per task: same move, different workload. Cost the thing you actually run, not the sticker.
The formula, in one line#
There are two numbers to compare. The API cost is trivial:
api_cost_per_month = monthly_tokens × api_rate_per_M ÷ 1_000_000The self-host cost is a fixed hourly rental times how many hours you keep the card, plus ops:
selfhost_cost_per_month = gpu_$per_hr × hours_run + ops_$Set them equal and solve for the volume where they cross — your break-even token volume:
breakeven_tokens = selfhost_cost_per_month ÷ (api_rate_per_M ÷ 1_000_000)That's the whole worksheet. The subtlety is entirely in hours_run: a GPU you rent 24/7 costs the same whether it embeds 30 billion tokens or thirty. Which brings in the second equation nobody puts on the pricing page — capacity:
max_tokens_per_month = throughput_tok_per_sec × 3600 × hours_run × utilizationSelf-host wins on cost only when your volume is above breakeven_tokens and your GPU is busy enough that you're not paying for idle hours. Those two facts fight each other, and that fight is the entire decision.
The numbers, grounded#
Treat all of these as as-listed in mid-2026 — verify before you commit. Prices move monthly.
Embeddings APIs, per million tokens:
| API | $/M (standard) | Dims | Notes |
|---|---|---|---|
OpenAI text-embedding-3-small | $0.02 | 1536 | batch ~$0.01; MTEB ~62 |
Voyage voyage-4-lite | $0.02 | — | 200M free tokens on the v4 gen |
| Jina v3 | $0.02 | 1024 | commercial API tier |
Cohere embed v3 | $0.10 | 1024 | light variant ~$0.02 |
Voyage voyage-4-large | $0.12 | — | premium quality tier |
OpenAI text-embedding-3-large | $0.13 | 3072 | batch ~$0.065 |
Open models you'd self-host (all run through a server like TEI, Infinity, or vLLM):
| Model | Dims | License | Position |
|---|---|---|---|
BAAI bge-m3 | 1024 | MIT | dense+sparse+multi-vector, 100+ langs; the workhorse |
nomic-embed-text-v2 | 768 (MoE) | Apache-2 | fast, catalog-scale ingest |
Qwen3-Embedding-8B | 1024 (32–1024 MRL) | Apache-2 | top open-weight MTEB (~75), but heavier/slower |
See Qwen3-Embedding vs EmbeddingGemma vs BGE-M3 for the quality trade and Voyage vs OpenAI vs Cohere vs Gemini for the API side.
GPU rental, per hour on commodity clouds: L40S ~$0.79, A10 ~$1.29, A100 80GB ~$1.19–1.39, H100 PCIe ~$1.99, L4 in the ~$0.40–0.80 range. Cross-shop these — see CoreWeave vs Lambda vs Nebius.
A worked example#
Take the honest matchup: BGE-M3 on a rented L40S at ~$0.80/hr, kept warm 24/7 (you have to, or cold starts eat your latency). That's $0.80 × 730 = ~$580/month fixed.
Now the break-even against each API tier:
| You'd otherwise pay | API $/M | Break-even tokens/mo | GPU duty needed to get there |
|---|---|---|---|
| 3-small / Voyage-lite / Jina | $0.02 | ~29B | ~28% |
| Cohere v3 | $0.10 | ~5.8B | ~6% |
| 3-large | $0.13 | ~4.5B | ~4% |
The "duty needed" column is the reality check. That L40S, running BGE-M3 at a working figure of ~40k tokens/sec, could in theory produce ~105B tokens/month at 100% utilization. So to beat the $0.02/M API you must keep it averaging ~28% busy every hour of the month — a real, steady ingestion pipeline, not a query embedded here and there. To beat the $0.13/M premium API you only need ~4% duty, which almost any always-on backfill clears.
And the punchline number: a saturated L40S embeds at
$0.80/hr ÷ (40,000 tok/s × 3600 s ÷ 1e6) = $0.80 ÷ 144 = ~$0.0056 per million tokensThat's 3.5x cheaper than the cheapest API — but only at the top of the utilization curve. At 28% duty your effective cost is exactly the $0.02/M you were trying to beat. Self-hosting isn't cheaper hardware; it's a bet that you can keep the card fed.
An idle GPU is the most expensive embeddings in the world. You pay by the hour whether or not you send it work — so the only question that matters is what fraction of those paid hours you actually use.
The one exception that flips everything: if you already own the GPU, the fixed rental disappears and the marginal cost is electricity — an L40S at ~350W is roughly $40/month. Now break-even against the $0.02/M API drops to ~2B tokens, and against the $0.13/M API to ~0.3B. A sunk-cost GPU is where self-hosting quietly wins for a lot of solo builders.
The hidden costs of self-hosting#
The $580 is the easy line. Three costs never show up in it:
- Ops time. You now run an inference server: deploy it, monitor it, patch it, and scale it under load. That's hours of your week, and your time is the scarcest input a solo founder has. The API's number is zero-ops; price your hours against the gap.
- Cold starts. A serverless GPU takes 10–60 seconds to load weights. If you scale to zero to save money, the first request after idle is unusable — so you keep a card warm, which is exactly the 24/7 rental the break-even assumes. Scaling to zero and low latency are mutually exclusive here.
- Re-embedding on model change. The dimensions and vector space are model-specific, so switching models means re-encoding your entire corpus — a one-time burst that can dwarf a month of steady-state embedding. Plan it before you pick a model (how to migrate embedding models in production), and cache aggressively so you never re-embed unchanged text on either side of the decision.
The founder move#
Run the division with your real volume and your real GPU quote. If you're under the break-even — and most solo apps embedding bursty user queries are, by a wide margin — the API is cheaper and zero-ops, and the decision is made. This mirrors the self-hosting cost breakdown for agents: the compute is rarely the reason.
If you're above it, or climbing toward it on a steady ingestion pipeline, self-host — and the moment you commit, the game becomes utilization, because a card at 28% duty saved you nothing. And if the deciding factor was never cost — you need embeddings back in 15ms inline, or the data can't leave your VPC, or the provider throttles your backfill — then skip the worksheet entirely. Those three are the real reasons to own the GPU. The token math just tells you what the privilege costs.
