The one-line version: Meituan's new VitaBench 2.0 asks whether an agent can learn a person over days and weeks of scattered conversations — and the best model in the paper, Claude-Opus-4.6 in thinking mode, scores about 0.50 with the entire history in front of it. Worse for anyone building on memory: the moment you replace that full history with a real memory layer — agentic or RAG — the score goes down. If your product line includes "it remembers you," this benchmark is the reality check to read first.

What VitaBench 2.0 measures that the last generation didn't#

The original VitaBench (accepted to ICLR 2026) put agents in a busy, tool-dense environment and asked them to complete complex one-shot tasks — we covered why even the best agents scored around a third in VitaBench: why the best agents score 32%. VitaBench 2.0 keeps the realism and adds the axis that actually defines personalization: time.

Instead of one hard request, an agent now faces 56 simulated users and 771 subtasks — about 14 per user — across three real-world domains: food delivery, in-store consumption, and online travel. To do well, it has to infer, use, and update a user's preferences across fragmented conversations and behaviors that span days, weeks, and months (repo). Scores are reported Avg@4 — the average success rate over four independent rollouts, so a single lucky run can't carry the number.

That framing is the point. "Personalization" as a product promise isn't "answer a question about a fact buried 20 turns ago." It's "know that this user switched to decaf last month, books aisle seats, and hates surge pricing — and act on it without being told again." VitaBench 2.0 is the first widely-shared benchmark aimed squarely at that.

The headline: a coin flip, at the ceiling#

Here is the number to sit with. Under the Full Context setting — the whole interaction history pasted into the prompt, the most generous configuration the benchmark offers — the strongest model reported is Claude-Opus-4.6 (thinking) at 0.503 Avg@4, with DeepSeek-V4-Pro (non-thinking) at 0.456 (repo results).

Full Context is the ceiling, not the floor. Nothing is hidden; the model isn't being asked to remember, only to read and act. And the best one still clears barely half. Long-horizon personalization, done right, is roughly a coin flip for the frontier — which means any confident "our agent learns you" claim is currently ahead of what the models can deliver.

The finding that should move a roadmap: memory made it worse#

Most benchmarks stop at "SOTA is X%." This one has a result that's directly actionable, and it's counterintuitive: most models degrade under realistic memory settings compared to Full Context — and it holds for both agentic memory and pipeline RAG memory (repo).

Read that twice. When the model manages its own memory — deciding what's worth storing and fetching it back later — it scores lower than if you'd just handed it everything. When a retrieval pipeline pulls "relevant" facts into the prompt, same story. Both approaches are lossy compression, and compression drops the one detail a later task turns out to hinge on.

Adding a memory layer to a personalization agent, on this benchmark, is not an accuracy upgrade. It's an accuracy tax you pay for lower token cost and latency — and if you didn't measure it, you're paying it blind.

This is not a fluke of one benchmark. It's the same result LoCoMo and LongMemEval already showed on one-shot memory — a full-context baseline beats the memory pipeline on accuracy — and the standard rebuttal was that those tests were too short to stress memory. VitaBench 2.0 removes that excuse: it is explicitly long-term and multi-session, exactly the regime memory layers are sold for, and full context still wins. The uncomfortable conclusion is that memory design is genuinely unsolved, not that the old benchmarks were soft. (For why detail-preservation beats fact-compression here, see the cost-performance analysis of memory vs long context.)

What a founder should actually do#

You don't get to opt out of memory — full context stops being an option the moment a user's history outgrows the window, and it's expensive long before that. So the move isn't "never use a memory layer." It's stop assuming it helps accuracy, and measure.

  1. Baseline against paste-everything. Before you ship a "remembers you" feature, run your memory layer and a plain full-context baseline on your own tasks. If memory isn't beating full context on accuracy, you've built a cost-and-latency optimization — a real and valuable thing — so price and market it as one, not as intelligence.
  2. Treat proactivity as experimental. "Anticipates your needs" is a research frontier here, not a guarantee. Keep a human-confirmation step on any ambiguous, irreversible action; the benchmark says the model will guess wrong often enough to matter.
  3. **Spend your engineering on memory design, not model choice.** The gap between Full Context and realistic memory is where your points are hiding — what you store, when you update it, and what you retrieve. That's the part a lab won't solve for you, and it's the part you own.

The one-line takeaway for the roadmap: personalization is a memory-design problem, and the benchmark that finally tests it says nobody has solved it yet. That's not a reason to wait — it's a reason to measure honestly, ship the cost story you can actually defend, and put your scarce engineering where the score says it counts.