Before a founder picks a model, a framework, or a price tier, they make a quieter decision that shapes everything downstream: does the agent run while the user waits, or does it run after the user leaves? Get this right and the rest of the stack follows. Get it wrong and you build the product twice.
Here's the whole decision in one screen, then the parts that trip people up.
The two shapes#
A synchronous agent runs while the user is watching. They send a message, tokens stream back, they read the answer, they act on it. Chat assistants, inline code completion, a RAG search box — the user is present for the whole run. State lives in memory for one request, and when the request ends, so does the agent.
A background agent runs after the user walks away. They assign a task — a ticket, a GitHub issue, a prompt — and go do something else. Minutes or hours later, the agent pulls them back with a pull request, a Slack ping, or a webhook. Devin, Cursor's Cloud Agents, OpenAI's Codex Cloud, GitHub Copilot's coding agent, and Claude Code on the web are all this shape: task in, work alone in an isolated cloud environment, PR out.
The variable that actually decides it#
Most teams reach for task length as the deciding factor — long tasks go to the background, short ones stay synchronous. That's a weak proxy, and it misleads constantly.
The question isn't how long the task takes. It's whether the user's next action depends on the result. If they'll sit and watch it land, go synchronous. If they'll go do something else and come back, go background.
Some twenty-second tasks belong in the background — kick off a report, keep scrolling, get pinged when it's ready. Some three-minute tasks belong in the foreground — the user is blocked, staring at the screen, unable to proceed without the answer, and sending them away would be cruel. Rank on dependency, not seconds. Length only tells you how hard the engineering will be, not which shape to pick.
Why background is a different product, not a flag#
The trap is thinking background is a checkbox on top of synchronous. It isn't. A synchronous agent holds its entire state in memory for one open connection and streams the result back over it — the model call is the product.
A background agent outlives the request that started it, and that single fact drags in a whole stack:
- a durable runtime and job queue to run work nobody is watching — which is its own managed-runtime-vs-self-host decision;
- a store that persists agent state across crashes and multi-hour pauses;
- a return address — the PR, the notification, the webhook (versus polling) — to reach a user who has long since closed the tab;
- a review surface where the human inspects and approves what the agent did.
The model call is the easy part of a background agent. The return address and the review surface are where the real product lives — and they're exactly what a synchronous prototype never made you build.
Cost inverts, and so do the guardrails#
The two shapes fail in opposite directions, so they need opposite guardrails.
A synchronous agent only burns money while someone is actively watching, so idle cost is zero — but every concurrent user demands low latency at the same moment, so a traffic spike hits you as a latency and capacity crisis. Budget for headroom and graceful degradation.
A background agent lets you queue, batch, and rate-limit, which smooths utilization beautifully — but it runs unsupervised, so a looping or stuck agent will happily burn tokens and compute with nobody watching. Background agents live or die on hard per-run spend caps, step limits, and timeouts. A synchronous agent that hangs annoys one user; a background agent that loops sends you a bill.
The decision, and the one-way door#
Decide by dependency, up front:
- The user's next action depends on the result, and they'll wait for it → synchronous. Stream tokens, keep them engaged, don't over-build.
- The user assigns it and leaves → background. Invest in the queue, the durable state, the return address, and the review surface — that's the product.
- Honestly, sometimes both → build background first. It's a superset: a background agent can always stream its progress to a user who happens to be watching, giving you the synchronous experience as a special case. The reverse isn't true — a synchronous agent can't survive a closed tab, so retrofitting background means adding the entire durable stack later, usually as a second execution path you maintain forever.
The most expensive version of this mistake is the common one: ship synchronous because it's faster to build, watch users ask to "just let it run," and bolt background on under deadline. Pick the shape from the dependency, not from what's easy this week — and if you're not sure, remember which direction the one-way door swings.
