Here is the structural problem with selling anything built on an LLM: your cost is a variable that moves with every token, and your price is almost always a fixed number a customer agreed to once. Most other software has this luxury too — the marginal cost of one more API call is basically zero — but AI features don't. Each request has a real, non-trivial, usage-scaled cost, and if you price like a normal SaaS company, the gap between your flat price and your floating cost is exactly where the money leaks out.
The short answer: compute gross margin as (price − token cost − other COGS) ÷ price, and remember that the token-cost term is the only one that scales with how hard a customer uses you. Price against your heaviest accounts, not your average one — because with AI, the average is a comforting lie and the tail is where you go underwater.
The one formula, and the one term that matters#
Gross margin on an AI feature is the ordinary formula:
margin % = (price − COGS) / price
COGS = token_cost + per_request_infra
token_cost = input_tokens × input_rate + output_tokens × output_rateEverything except token_cost is roughly fixed per request. token_cost is the live wire. And two properties of it trip up founders:
- Output is the expensive half. Across current-generation Claude models at July 2026 list price, output tokens cost about 5× input tokens — Sonnet 4.6 is on the order of $3 per million input tokens and $15 per million output, Haiku 4.5 roughly $1 and $5. A feature that generates long responses is dominated by its output, so a "cheap" model with verbose answers can cost more than an "expensive" one that's terse.
- It scales with the customer, not with you. Your rent, your salaries, your fixed infra don't care whether a given account sends ten requests or ten thousand.
token_costdoes. That single fact is why AI unit economics behave unlike normal SaaS.
Work one example. Say your product turns a prompt into a structured report, and a typical report is 4,000 input tokens and 2,000 output tokens on Sonnet. That's 4000/1e6 × $3 + 2000/1e6 × $15 = $0.012 + $0.030 = $0.042 per report — about four cents. Charge $0.50 a report and you're at ~92% gross margin. Comfortable. Until you meet the customer who runs 300 reports a day and is on your $99/month flat plan.
The trap: you price for the average, you pay for the tail#
Token spend per account is almost never uniform — it's long-tailed. A handful of power users each burn many times the median. Picture the same report product on a flat $99/month "unlimited" plan:
- The median user runs 200 reports a month. Cost:
200 × $0.042 = $8.40. Margin: excellent. - The p95 user runs 4,000 reports a month. Cost:
4,000 × $0.042 = $168. On a $99 plan, that account loses you $69 every month — before support, before infra, before anything else.
And here's the cruelty: the p95 user is getting enormous value, so they are the least likely to churn. Flat pricing doesn't just lose money on the tail — it retains the accounts it loses money on and sheds the cheap ones. You have built a machine that selects for your most expensive customers.
Pricing for the average is the default mistake because the average looks healthy in the dashboard. The heavy tail is where the P&L actually lives.
Flat pricing on a usage-scaled cost is a bet that no customer will use you as much as they possibly can. The customers who love you most will take that bet every time.
Four models that survive a power user#
The fix is to let the variable cost show up somewhere in the price — or to cap it. Four shapes, from the compare table above:
- Usage-based. Meter the unit of work and bill it. Cost and price move together, so a power user is a bigger customer, not a worse one. The cleanest alignment; the hardest to make feel predictable.
- Hybrid (base + overage). A base fee covers the platform and an included allowance; usage above it is metered. This is the workhorse for B2B AI: predictable revenue floor, and the tail pays its own way. Most products should start here.
- Credits / prepaid. The customer funds a balance that usage draws down. You literally cannot spend money you weren't paid — margin can't go negative because the ceiling is the customer's wallet.
- Flat — but only with a real cap. Flat pricing is fine if you enforce a hard usage ceiling (a monthly quota, a rate limit, credits behind the scenes). "Unlimited" is a marketing word, not a pricing model. If you say it, you'd better have measured your p99 and be able to afford them. A related pattern is splitting a flat seat from a metered pool, the way Cursor's team plans separate a premium seat from usage.
Whichever you pick, meter the unit of value, not raw tokens. Customers understand "per report," "per agent run," "per 1,000 actions." They do not want to think in tokens, and pinning your price to tokens couples it to a model choice you might change next quarter. Size the value-unit's token cost with a buffer, price the unit, and keep the token accounting on your side of the wall.
Before you reprice, drop the floor#
Repricing is slow and it annoys customers. Cutting cost is fast and invisible. Do the cost work first — every cent off the floor is margin you keep on every account you already have:
- Prompt caching. If a big, static chunk of your prompt repeats across requests — a system prompt, a rules document, few-shot examples — cache it. Cache hits run at roughly 10% of the base input rate, so a cacheable prefix that dominates your input can cut input cost by an order of magnitude. The mechanics are in how to cut your Claude API bill with prompt caching; the interaction with context editing is in prompt caching vs context editing.
- Batch the non-urgent. Work that doesn't need a synchronous answer — overnight report generation, bulk enrichment, evals — runs at about half price on batch endpoints. If a chunk of your COGS isn't latency-sensitive, that's a ~50% cut on that chunk for free.
- Right-size the model per task. Output dominance means the cheapest path is often a smaller model with tighter output, not a bigger model you trust to be brief. Route easy requests to Haiku-class models and reserve the flagship for the calls that need it.
- Cap the runaway account. Independently of pricing, put a technical ceiling on per-account consumption so a bug, an abuser, or an over-eager power user can't run your margin negative before you notice — the same discipline as throttling an agent against a third-party rate limit.
The rule to price by#
Set your price against your p95 cost per account, not your average — and leave a buffer above it for the heavier user you haven't onboarded yet. The average tells you whether the business looks good this month; the tail tells you whether it survives its best customers. Get the cost floor down with caching and batching, choose a model that lets the variable cost surface in the price or a cap that contains it, and meter the value-unit rather than the token. Do that and a power user becomes the best thing that can happen to you instead of the account you quietly dread. For the deeper build-vs-buy version of the same math, see self-hosting inference vs API cost.
