What Should an AI Agent's Tools Return? Designing Tool Results for the Context Window
Everyone tunes a tool's inputs — name, schema, description. The likelier production failure is the output: the right tool returns a payload that floods the model's context window.
4 min
When Should an AI Agent Compact Its Own Context? The Case Against Fixed Thresholds
Most agents summarize their context when a token counter trips. A 2026 result argues the counter is the wrong trigger — and that letting the model decide is both cheaper and more accurate.
4 min
Resumable LLM Streaming: How to Survive a Refresh Without Repaying for the Answer
SSE hands you a Last-Event-ID header that looks like free stream resumption. It isn't — it's a cursor with nothing behind it. The real fix is the one decision everything else follows from.
6 min
Request Hedging for LLM Tail Latency: Race the Slow Call, Don't Retry It
Every other latency fix speeds up the typical request. Hedging is the only one that attacks the slow tail — by firing a duplicate after your p95 and taking whichever finishes first.
5 min
RAFT vs RAG vs Fine-Tuning: When to Train on the Documents You Retrieve
RAG gives the model an open book; fine-tuning makes it memorize. RAFT does the thing neither does — it trains the model on bad retrieval, so it survives the wrong chunk your production retriever will hand it.
4 min
OpenTelemetry GenAI Semantic Conventions: The Spec Your Observability Tool Depends On Is Still 'Development'
Every LLM-tracing vendor now sells the same promise — open, portable, OTel-native. The schema that makes that true isn't finished, and there's an env var to prove it.
4 min
MMR vs Reranking in RAG: Why Your Top-K Returns the Same Fact Five Times
A reranker and a diversity step look like the same 'advanced RAG' upgrade. They fix opposite failures — and the benchmark that everyone cites quietly shows that turning on diversity often does nothing at all.
5 min
Mixture of Agents vs a Single Model: Why Ensembling LLMs Usually Loses to Sampling One Good Model Twice
Mixture-of-Agents wins by quality, not by variety — and a careful 2025 replication found that aggregating repeated samples from your single best model beats mixing different ones in most cases. Here's when an ensemble actually pays, and when it just adds latency.
5 min
MCP Tasks: How Long-Running Agent Work Survives a Stateless Server
The 2026-07-28 spec made MCP stateless. Long-running work and statelessness are in direct tension — and the Tasks extension resolves it by handing the bookkeeping to the client. The tell is what got deleted.
5 min
MCP Extensions, Explained: How the 2026 Spec Grows Without Breaking the Core
The next Model Context Protocol release stops adding features to the core and starts subtracting them. The Extensions framework is how — and 'in the spec' no longer means 'in the core.'
5 min
Implicit vs Explicit Prompt Caching: When to Pay for a Cache You Control
Both kinds of cache hit read at the same discount, so cost-per-hit is the wrong thing to choose on. The real split is a guarantee you pay for versus a freebie you can't shape.
5 min
Streaming Structured Output From an LLM: How to Render JSON Before It's Done
A JSON object isn't valid until its closing brace — but your UI shouldn't wait for it. The trick is realizing a streamed object is a view, not a value, and validating it exactly once: at the end.
5 min
How to Set a Timeout for an AI Agent: A Per-Call Timeout Won't Bound the Loop
The SDK's 10-minute default times out one call; an agent makes dozens. You need a deadline the whole loop shares — and cancelling to enforce it still costs tokens and can corrupt state.
6 min
How to Roll Back an AI Agent's Actions: The Saga Pattern for Tools That Can't Undo
An agent has no ROLLBACK: when step three fails, the first two already happened in the world. The fix is a compensating undo for every tool — and putting the one you can't undo last.
4 min
How to Monitor an AI Agent in Production
Your agent can be HTTP-200, fast, and cheap while being completely wrong. The metrics that keep a web app healthy are blind to the ways an agent actually fails.
5 min
How to Load-Test an LLM App: You're Stress-Testing the Rate Limiter, Not the Model
For an app built on a hosted LLM API, the wall you hit under load isn't the model's speed — it's the provider's rate limiter and your own retry policy. Test for the ceiling and the fall, not the throughput.
5 min
How to Handle Tool Errors in an AI Agent: Return the Failure, Don't Raise It
The try/except instinct that keeps a normal program alive is the one that kills an agent. A tool error isn't an exception to catch — it's the next message in the conversation, and where you put it decides whether the agent can recover.
6 min
How to Evaluate a Reranker for RAG: The Number That Caps It Isn't the Reranker's
A reranker can only reorder what your retriever already fetched, so the ceiling on its lift is your stage-one recall — measure that first, then judge the reranker as the latency and dollars you pay to convert recall into precision.
5 min
How to Do RAG Over Tables: When to Embed Rows and When to Generate SQL
Your RAG pipeline works on documents and falls apart on a spreadsheet — because a table's meaning lives in its grid, and an embedding flattens the grid away.
5 min
How to Deploy an AI Agent to Production
An agent isn't a stateless web service — it's a long-running, resumable process. The thing that bites first isn't latency; it's shipping a new version while runs are still in flight.
5 min
GAIA2: The Agent Benchmark Where the Clock Never Stops
Static benchmarks freeze the world while an agent thinks. Meta's GAIA2 lets time run — and the smartest model, GPT-5, turns out to be the one that misses deadlines.
4 min
Does an AGENTS.md File Actually Make Your Coding Agent Better?
The first rigorous benchmark of repository context files is in, and the answer is uncomfortable: the auto-generated ones make agents slightly worse, the hand-written ones barely help, and both raise your bill ~20%.
5 min
Disaggregated LLM Inference: Why Prefill and Decode Are Moving to Separate GPUs
The two halves of every LLM request fight each other on the same GPU. Disaggregated serving splits them onto separate hardware — and the win is real, but only past a certain scale.
4 min
Cost-Aware Agent Evaluation: Why Your Benchmark Needs a Dollar Axis
An agent leaderboard that ranks only on accuracy is secretly ranking on willingness to spend. Add the cost axis and the board's #1 is often not even on the frontier.
5 min
Reward Hacking in AI Agents: When the Eval Becomes the Attack Surface
If your agent's reward is a number it can reach without doing the work, it will eventually reach the number without doing the work — and 2026's research says that habit doesn't stay contained.
5 min
Who Controls MCP Now? Inside the Agentic AI Foundation
For a year the question that stalled enterprise bets on MCP was 'what happens when Anthropic changes its mind?' In December that question got an answer — and the answer reveals what the standards war was really about.
5 min
WebMCP vs MCP: Why Browser Agents Get Their Tools From the Page
A new web standard lets a website hand an AI agent a typed menu of its own functions — no server, no OAuth. The catch is hiding in that 'no OAuth.'
5 min
Vercel eve vs LangGraph: Library You Host, or Harness You Rent
Vercel's new agent framework treats an agent as a directory of files. LangGraph hands you a portable graph. The decision isn't the loop they run — it's who owns the production stack wrapped around it.
5 min
AWS Trainium vs NVIDIA GPU for LLM Inference: The Bill Is Cheaper, the Onramp Isn't
Trainium2 and Inferentia2 sell real price-performance and AWS capacity. NVIDIA sells CUDA. The decision is whether the Neuron SDK supports your model and serving stack — and how much engineering you'll spend finding out.
5 min
Tool-Result Caching for AI Agents: The One Cache That Can Be Wrong
Prompt and semantic caches store the model's work and fail cheaply. Tool-result caching stores the world's — and it forces a question every agent codebase has dodged: which tools are safe to cache?
5 min