The short version: Google Cloud's Always-On Memory Agent gives an agent durable memory with no vector database and no embeddings. An LLM (Gemini 3.1 Flash-Lite, via Google's ADK) runs continuously in the background, consolidates what the agent has learned, and writes structured records into plain SQLite. It beats a RAG pipeline when you have one agent with one evolving context and freshness matters; RAG still wins when you must retrieve over a large, mostly-static corpus. The deciding factor is corpus size times update frequency: small-and-changing favors consolidation, large-and-static favors retrieval.
Two mechanisms for the same job#
Both approaches answer the same question — how does an agent remember things across turns and sessions? — but they put the work in different places.
A RAG pipeline does the work at query time. You chunk your data, embed each chunk into a vector, and store the vectors. When a question arrives, you embed the question, find the top-k nearest chunks, and stuff them into the prompt. The memory is passive: it sits in the index until a query reaches in and pulls the closest matches. Storage is cheap; every query pays for retrieval, and every change to the source data pays to re-embed.
Google's Always-On Memory Agent does the work ahead of time. An LLM runs as a continuous process — not a one-shot call — reading new information, reconciling it against what's already stored, and writing consolidated, structured records into SQLite. There is no embedding step and no vector store. By the time the agent needs to answer, the memory is already synthesized, not a pile of chunks waiting to be ranked. The tradeoff is explicit: you pay for background inference around the clock (which is why the design uses a cheap, low-latency model) instead of paying at each query.
RAG asks its question of a frozen index. Continuous memory keeps thinking between the questions.
When continuous memory wins#
The consolidation model shines when the memory is small enough to hold and coherent enough to matter. Concretely:
- A single agent with one evolving context. A personal copilot, a per-customer support thread, a founder's own long-running assistant. There's one story to keep straight, and keeping it straight is the whole job.
- Freshness beats breadth. When the value is in reflecting what just happened — the decision made an hour ago, the preference stated yesterday — background consolidation keeps memory current in a way a nightly re-index can't.
- You want to drop infrastructure. No vector database to run, no embedding model to call, no chunking strategy to tune. For a bootstrapped team, "one fewer moving part" is a feature you can feel in the ops bill.
This is the case Google's reference architecture is built for, and it's a common one: most agents in production serve one user or one tenant at a time, over a working set that fits comfortably in what a cheap model can consolidate.
When RAG still wins#
Retrieval isn't going anywhere for the workloads it was built for:
- Large, mostly-static corpora. Thousands of product docs, a company knowledge base, a legal archive. Continuously re-consolidating all of it with an LLM would be wasteful or flatly impossible; approximate nearest-neighbor search over embeddings is exactly the right tool.
- Many tenants, isolated data. When you're retrieving across separate customers' data, you want a queryable index per tenant, not one consolidation loop trying to hold every tenant's context at once.
- Auditability of sources. RAG hands you the exact chunks it retrieved, which you can cite and show. A consolidated memory record is synthesized, so provenance is fuzzier unless you engineer it in.
The honest framing is corpus size × update frequency. Small and changing → consolidate. Large and static → retrieve. The awkward middle — large and changing — is where you end up running both: RAG over the stable corpus, a consolidation loop over the hot, per-agent working set.
The founder's read#
Don't reach for a vector database by reflex. If you're building a single-context agent — and most first products are — Google's architecture is a standing argument that you can ship durable memory with an LLM loop and a SQLite file, and add embeddings later only when the working set outgrows what the model can hold. Start with the cheaper mechanism, and let the corpus tell you when you've earned the vector store.
For the week's other moves — Alibaba's agent-native cloud and Kimi K3's open weights among them — see The Founder's Wire, week of July 23.
