MON, NOVEMBER 16, 2026
Independent · In‑Depth · Practitioner‑Tested
LLMs

Memorilla vs MemoRAG vs xRAG: LLM Memory Compared (2026)

Sixteen soft tokens against retrieval, compression and reading the whole thing.

🕐 8 min read 👁 22 views 📅 Nov 16, 2026
Straight answer
● Winner on average Memorilla, 49.68 against 41.92 for xRAG and 38.02 for Mem0.
● Biggest gap PMv2 short answer: 44.60% against 30.90% for xRAG.
● It loses three NarrativeQA and PV4 to MemoRAG, PubMedQA to LLMLingua-2.
● The real headline It beats reading the whole document, on a hundredth of the tokens.

Memorilla is not competing with retrieval on recall. It is competing with full-context prompting - putting the entire document in the prompt and letting the model read it - and beating that while reading roughly one percent of the tokens is the result that matters. Here is where each approach actually wins.

The benchmark table

BenchmarkMemorillaNext bestDeltaNote
FactKG (accuracy)84.33%80.43% full context+3.90108x fewer tokens
TriviaQA (exact match)57.0056.60 MemoRAG+0.4052.60 full context
PMv2 (short answer)44.60%30.90% xRAG+13.70widest margin
LaMP-7 (ROUGE-L)48.3746.84 xRAG+1.53
LaMP-4 (ROUGE-L)18.1514.21 full context+3.94474x fewer tokens

Where Memorilla loses

Three benchmarks, and the pattern in them is useful. MemoRAG leads on NarrativeQA (21.80 against 19.68 ROUGE-1) and PV4 (67.88% against 66.65%). LLMLingua-2 leads PubMedQA by a wide margin, 69.00% against 59.00%.

Narrative comprehension and clinical question answering are both cases where the exact wording carries the meaning. Compressing a document into 16 soft tokens is lossy by design, and what it loses first is phrasing. If your task turns on how something was said rather than what was stated, retrieval that returns the original passage still wins.

What each approach actually is

Full context puts everything in the prompt. Perfect fidelity, linear cost, and it hits a window limit. MemoRAG and classic retrieval fetch passages and insert them - the model reads real text, so wording survives, but recall depends on the retriever. xRAG and LLMLingua-2 compress the context before the model sees it. Mem0 maintains a memory store across turns. Memorilla is different in kind: it encodes documents once with a frozen embedding model, then a trained module emits 16 soft tokens directly in the decoder's embedding space. Nothing the model reads is text.

That is why only the memory module trains - about 194 million parameters - while the decoder, a frozen Qwen3-8B, is untouched. You are not fine-tuning a model, you are giving it a different kind of input.

The continual-learning result

On TextWorld, with the memory module trained by GRPO and only 8 memory tokens, average success rate rose from 0.186 to 0.242 and pass@3 from 0.250 to 0.375, against full-history in-context learning. That is the more interesting half of the paper: an agent that accumulates experience without its context growing.

Which to use

By task
● Factual recall over many documents Memorilla. Widest margins, lowest token cost.
● Narrative or clinical text MemoRAG or plain retrieval. Wording survives.
● One document, fits the window Full context. Nothing beats reading it.
● Agent that learns across episodes Memorilla with GRPO. The TextWorld result.

The practical read: if your cost is dominated by stuffing the same documents into every prompt, this is the first compression method that beats not compressing at all on most of what was tested. If your answers depend on exact phrasing, it is the wrong tool and retrieval still has the job. Our launch coverage has the release detail.

FAQ

Is Memorilla better than RAG?
On factual recall benchmarks, yes - it averages 49.68 against 41.92 for xRAG and 38.02 for Mem0. On narrative comprehension MemoRAG still leads NarrativeQA and PV4, and LLMLingua-2 leads PubMedQA, so it depends on whether exact wording matters to your task.
Does Memorilla beat putting the whole document in the prompt?
On most benchmarks tested. FactKG 84.33% against 80.43% using 108 times fewer document tokens, and LaMP-4 ahead by 3.94 ROUGE-L using 474 times fewer.
How many memory tokens does Memorilla use?
16 in the main experiments, and 8 in the TextWorld continual-learning setting.
Do you need to retrain the language model?
No. The decoder stays frozen - the paper uses Qwen3-8B - and only the memory module of roughly 194 million parameters is trained.
When should you not use it?
When the answer depends on exact phrasing. Compressing into soft tokens is lossy by design and loses wording first, which is why retrieval still leads on narrative and clinical text.
⚖ Our Verdict

Memorilla wins on factual recall across documents, averaging 49.68 against 41.92 for the next best compression method, and beats full-context prompting while reading about one percent of the tokens. Retrieval still leads where exact wording carries the meaning - MemoRAG on NarrativeQA and PV4, LLMLingua-2 on PubMedQA.