How to Use Kimi K3 Cheaply via API: Prompt Caching and the $0.52 Effective Price
The $3/M list price isn't what you actually pay. Kimi K3's cache-hit input is $0.30/M, and with the reported ~92% cache-hit rate the effective input cost lands near $0.52/M — but only if you structure prompts so the cache actually hits. Here's the copy-paste setup and the one ordering rule that decides your bill.
4 min
Context Rot: The Research Explaining Why a 1M-Token Window Doesn't Give You a Million Usable Tokens
Two 2025 studies put real numbers on a thing every builder half-knew: models degrade long before their advertised context limit — and worst exactly when the answer needs a little reasoning. The window on the box is a storage spec, not a performance spec.
4 min
Cheap 1M-Token Context Just Landed. Do You Still Need to Manage Your Agent's Context?
Qwen3.7 Flash lists a 1M-token window at ~$0.03/$0.13 per million tokens. The tempting conclusion — stop compacting, just dump everything in — is half right. Cheap context fixes the bill. It does nothing for the rot.
4 min
Context Engineering vs Prompt Engineering: The Line Every Agent Builder Now Draws
Prompt engineering optimizes a string you write once. Context engineering optimizes a process that runs every turn. When agents went long-horizon, the bottleneck moved from what you say to what's in the window right now — and the job changed with it.
4 min
Does Context Editing Actually Save Money? Measure the Cache Cost, Not the Cleared Tokens
Context editing reports a big 'cleared_input_tokens' number and it feels like a win — but every clear invalidates your prompt cache, so the headline can hide a higher bill. Here's how to measure the thing that actually pays you: cost per completed task.
5 min
The Number That Decides Kimi K3 Self-Host vs API Isn't the GPU Bill — It's the Cache Hit
Every rent-vs-own analysis of the 2.8T open-weight model quotes the $3/$15 sticker and stops. For an agent, the real price is $0.30 — and that one number moves the break-even to 'basically never.'
5 min
How to Set Up Context Editing in the Claude API: A Copy-Paste Config for Long-Running Agents
Your agent slows and drifts as tool output piles up in the window. Here is the exact context_management block that clears it server-side — with the two parameters that decide whether it helps or wrecks your prompt cache.
4 min
How to Turn On Claude's Context Compaction — the Same compact_20260112 Switch on the API and on Bedrock
Compaction is one declarative edit that summarizes old turns automatically when your prompt gets big. The switch is identical on the Anthropic API and Amazon Bedrock — the only things that move are the request envelope and one billing number that hides the real cost.
4 min
The Context Engineering Playbook for Long-Running Agents: Write, Select, Compress, Isolate
Your agent doesn't fail because the model got dumb. It fails because you let its context window rot. Here is the four-move playbook — with the exact Claude API calls under each move.
5 min
Agentic Loops That Run for Hours: Checkpointing vs Context Management Are Two Different Problems
The moment a task outlives one context window, builders reach for a bigger prompt — and it fixes neither failure. A long-running loop dies two unrelated deaths, and each has its own cure.
4 min
How to Tune clear_at_least So Context Editing Doesn't Nuke Your Prompt Cache
Context editing deletes old tool results to keep your agent inside the window — but every clear invalidates the cache below it. The clear_at_least knob is the whole fix. Here's the break-even math and the config to set it right.
3 min
Subagents vs Compaction: When to Isolate a Long-Running Agent's Context Instead of Editing It
Context editing and compaction both fight a full window by damaging what's already in it. A subagent never lets the mess in — it gets a fresh window and hands back one clean result. Here's the line between them.
5 min
Prompt Caching vs Context Editing: One Cuts the Price, the Other Cuts the Count
They both live in your 'lower the agent's token bill' folder, so builders reach for them interchangeably. They aren't. One makes the tokens you keep re-sending cheaper; the other deletes tokens so you stop sending them — and they quietly fight over your cache.
4 min
The 84% and the 39%: What Anthropic's Context-Management Numbers Actually Measure
Anthropic says context editing cut tokens 84% and memory-plus-editing lifted task success 39%. Both figures are real. Neither says the model got smarter — they measure escaping a wall your agent may never hit, or may hit in a shape the benchmark never tested.
4 min
Tuning Claude's Context Editing: trigger, keep, and clear_at_least Without Wrecking Your Cache
Context editing keeps a long-running agent inside its window by clearing old tool results — but the defaults fire late and fight your prompt cache. Here are the four knobs that decide how often it clears, what survives, and whether each clear is worth the cache re-write.
4 min
The Founder's Wire, Week of July 22: Your Context Goes Portable, HR Agents Move Into the Chat Window, and an Open Coder Punches 10× Its Weight
Three shipping-this-week moves that all point the same direction — the agent stack is coming apart into swappable layers you own, not one vendor's bundle. What Creed, Netchex Mesh, and Poolside's Laguna S 2.1 mean for a founding team.
4 min
Change an Agent's Rules Mid-Run Without Blowing Up Your Prompt Cache: Mid-Conversation System Messages Are GA
You can now append a system instruction partway through a Claude conversation instead of editing the top-level system field — so a long agent can pick up a new rule after 40 cached turns without re-paying for all of them. Here's the API shape, the one placement rule that returns a 400, and why it's a direct token-cost win.
5 min
GPT-5.6 Rewired Prompt Caching: A Hands-On Guide to prompt_cache_options
The July 9 GA quietly changed the caching contract — explicit breakpoints, a mandatory cache key, a 30-minute floor, and one gotcha that silently skips the write exactly where agents want it most.
5 min
How to Cut Your Claude API Bill by up to 90% with Prompt Caching
If you send the same big system prompt, document, or tool list on every request, you're paying full price for it every time. Here's the four-line change that makes the repeated part cost a tenth as much — with the code, the pricing math, and the one bug that silently turns it off.
5 min
How to Handle a Tool Result Too Large for the Context Window: Truncate, Paginate, or Hand Back a Handle
The overflow that kills agents happens at the one boundary the MCP spec never paginated — the tool result. And the reflex fix, truncating to N characters, is the only option that's strictly worse than doing nothing.
5 min
Context Offloading for AI Agents: Writing Tool Results to Disk to Beat the Context Window
The counterintuitive fix for context bloat is to stop reading tool output. Offload the payload to a file, hand the model a pointer — and move the retrieval decision from write-time to read-time.
4 min
Optical Context Compression: When It's Cheaper to Show Your Agent a Picture of Its History
DeepSeek-OCR, Glyph, and AgentOCR all render text into images so a vision model can read more with fewer tokens. The compression is real — but a December rebuttal says the honest competitor isn't full text, it's just deleting the old stuff.
5 min
How to Version Prompts in Production AI Agents: A Prompt Change Is a Deploy
Every prompt tool sells the same feature — edit the prompt without shipping code. Stated precisely, that feature is: change production behavior with no PR, no eval run, and no pinned model. Here's how to keep the convenience without the shadow deploy.
5 min
Semantic Caching vs Prompt Caching: Which One Actually Cuts Your LLM Bill (and Which Can Return a Wrong Answer)
They both have 'caching' in the name and both promise to slash your token spend, but they cache different things at different layers with different safety profiles. One's worst case is a cache miss. The other's worst case is a confidently wrong answer.
4 min
How to Summarize a Document That Doesn't Fit in the Context Window: Map-Reduce vs Refine vs Not at All
Map-reduce's 'reduce' step quietly re-creates the exact overflow you were escaping. Refine can't parallelize. And in 2026 the fastest-improving option is often to stop summarizing and put the whole document in a million-token window — if you can pay the middle.
5 min
How to Write a System Prompt for an AI Agent
A chatbot's system prompt sets a personality. An agent's is control logic the model rereads on every turn of the loop. Stop writing a persona and write a policy.
5 min
When Should an AI Agent Compact Its Own Context? The Case Against Fixed Thresholds
Most agents summarize their context when a token counter trips. A 2026 result argues the counter is the wrong trigger — and that letting the model decide is both cheaper and more accurate.
4 min
Implicit vs Explicit Prompt Caching: When to Pay for a Cache You Control
Both kinds of cache hit read at the same discount, so cost-per-hit is the wrong thing to choose on. The real split is a guarantee you pay for versus a freebie you can't shape.
5 min
Tool-Result Caching for AI Agents: The One Cache That Can Be Wrong
Prompt and semantic caches store the model's work and fail cheaply. Tool-result caching stores the world's — and it forces a question every agent codebase has dodged: which tools are safe to cache?
5 min
RULER vs Needle-in-a-Haystack: How to Measure an LLM's Real Context Length
The number on the spec sheet is a memory allocation, not a comprehension score. A needle test passing at 1M tokens tells you the model can find a string — not that it can use the context. Here's the benchmark that measures the difference.
5 min
Prompt Format: JSON vs XML vs Markdown vs YAML — and Why Input and Output Want Opposite Things
The reflex is to wrap everything in JSON because it's 'structured.' On the way into a prompt that's a token tax; on the way out it's an accuracy tax. The right answer is split, not single.
4 min
Prompt Caching Pricing in 2026: Anthropic vs OpenAI vs Gemini vs Bedrock
Every provider now sells the same ~90% discount on repeated context. The number on the brochure is not where the bills actually diverge — three quieter terms are.
4 min
Context Editing vs Compaction vs the Memory Tool: Keeping a Long-Running Agent in Its Window
A long-running agent fails when its window fills with stale tool output. Anthropic ships three levers for that — and the trap is treating them as competitors instead of a division of labor.
7 min
Prefix Caching vs Prompt Caching: The Three LLM Caches Everyone Confuses
They share a word and almost nothing else. One discounts your bill, one reuses GPU memory, one can hand back the wrong answer — and teams keep enabling the one they didn't mean.
4 min
How to Manage Context in a Long-Running Agent: Clearing vs Compaction vs Memory
An agent that runs for a hundred turns will blow past any context window. The fix is three different mechanisms — and the order you reach for them is the opposite of most people's instinct.
4 min
GEPA vs MIPROv2: Why Reflective Prompt Optimization Beats More Samples
GEPA optimizes prompts by reading the agent's own failure traces in plain language instead of chasing a scalar score — and reports beating an RL baseline with up to 35x fewer rollouts.
5 min
Prompt Compression for LLM Agents: LLMLingua vs LLMLingua-2 vs Selective Context
Tools that shrink a prompt by 2–20x before it hits the model promise a smaller token bill. Whether you actually save anything depends on a comparison nobody runs first — compression versus caching.
4 min
Context Engineering for AI Agents: Managing the Attention Budget
Prompt engineering optimized a string. Context engineering manages a finite, decaying budget — because the context window is not a bucket you fill, it is attention that rots as it fills.
5 min
DSPy vs TextGrad vs AdalFlow: Optimizing Prompts Instead of Writing Them
Three Python libraries that treat your prompt as a parameter to be tuned, not a string to be hand-crafted. They disagree about what the optimizer needs from you — and that's the whole decision.
6 minLatest in Prompts & Optimization
Not buyer's guides — the news, teardowns, and explainers behind this topic.
