The short version, up front: the 2026 agent-memory leaderboards climbed — Mem0's repository now reports 94.4 on LongMemEval and 92.5 on LoCoMo, up from 67.8 and 71.4 a year ago — and got harder to trust, not easier. Directly beneath those numbers, Mem0's own README says the scores "reflect Mem0's managed platform, which includes proprietary optimizations not available in the open-source SDK." The bar on the chart is not the bar you get from pip install mem0ai. Everything below is how to read that gap — and the two numbers that actually transfer to your agent.

This is the 2026 follow-up to a rule that hasn't changed: there is still no shared grader for these benchmarks, so cross-vendor accuracy bars were never comparable. What's new this year is that the numbers went up while quietly moving further from anything you can reproduce.

1. The headline number is from a platform you can't install#

Read the fine print under any 2026 memory score and check one thing first: managed platform, or open-source SDK?

Mem0 publishes both a hosted, paid platform and an MIT-licensed package. The 92.5/94.4 figures on the repo are labeled as the managed platform's results, "which includes proprietary optimizations not available in the open-source SDK." That is an unusually honest disclosure, and it's the most important sentence in the whole comparison. It means the number that sells the product and the number you can verify are two different numbers — and only the lower, unstated one ships in the box you install.

So if your plan is to self-host the open-source SDK — which is most solo builders' plan — the headline is an upper bound you have not measured, not a spec you can count on. Treat it the way you'd treat a benchmark run on hardware you don't own.

2. Read the retrieval budget next to the accuracy#

The second number to find is the retrieval budget. Mem0's 2026 scores are run as "single-pass retrieval (one call, no agentic loops) at a top_200 retrieval budget." Single-pass is genuinely good news — no expensive agentic re-querying loop. But top_200 means up to 200 candidate memories are pulled into the model's context on every query.

That matters because it partly re-spends the savings. A memory layer's entire pitch is cost: compress a long transcript into a few atomic facts so you stop paying to re-read the whole history. Retrieve 200 memories per query and you've put a meaningful slice of that transcript back into context to buy the accuracy. The score and the budget have to be read together — a high accuracy at a large retrieval budget is a different product than a high accuracy at top_10.

3. The one benchmark that tests the regime memory is for#

LoCoMo conversations run about 16–26K tokens. That fits inside essentially every modern context window, so LoCoMo mostly asks whether a system can read a transcript it could have just pasted — a retrieval task, not a test of memory under pressure. VitaBench and LongMemEval's harder splits pushed on this, and the lesson held: on short histories, full context keeps winning.

BEAM is the 2026 answer to that critique. It stretches conversation length to 1M and 10M tokens — the beyond-the-window regime where a memory layer finally has to earn its keep, because no model can just hold the history. This is the benchmark to watch, and it's why the honest number isn't the headline.

Because look at what happens as the horizon grows. On Mem0's own BEAM figures: 64.1 at 1M tokens, 48.6 at 10M. Same system, longer memory, and accuracy falls by a quarter — down to roughly a coin flip on the hardest split. The 94% is a short-conversation number; the ~49% is the long-conversation number, and long conversations are the reason you were shopping for a memory layer in the first place. When a vendor reports both, the long one is the one that describes your production.

4. Stars measure ecosystem, not accuracy#

It's tempting to let adoption break the tie. Don't. As of now the open-source memory projects sort like this:

Stars track documentation, community, and momentum — genuine reasons to bet on a project's longevity. They do not track whether it remembers your users correctly. Note that Letta, a serious system, publishes no LoCoMo or LongMemEval bar at all, because its architecture is aimed at long-running self-editing agents rather than at topping a fact-recall chart. If you ranked by stars you'd never learn that. Rank by architecture fit and reproducible cost instead.

5. The two numbers that actually transfer#

Here's the part that survives every leaderboard revision. Stop trying to pick a winner from published bars and measure two things on your own data:

  1. Recall on your traces. Take a sample of your real conversation logs, ask the questions your users actually ask, and score whether the memory layer surfaces the right facts. Run a full-context baseline in the same harness — if pasting the whole history still wins at your conversation lengths, you may not need a memory layer yet, and that's a finding worth having before you add a dependency.
  2. Tokens and p95 latency per query — measured on the open-source package you'll deploy, not the managed demo, at your real history sizes. This is the axis a memory layer is supposed to win, and it's the one that's honest to compare.

Those two numbers are portable: they describe your agent, your traffic, and the code you actually ship. A vendor's managed-platform accuracy bar describes none of those. The 2026 leaderboards got taller and glossier this year; the thing that decides your build is still a fifty-line eval script pointed at your own logs. If you want the framework-by-framework version of that decision, we walk it here — but bring your own recall-and-cost run to it, because that's the evidence the charts can't give you.