Study notes: How LLMs Could Place Explicit Memories

Train a model on text interleaved with what a retriever would have fetched, and the model starts to produce that material itself. Spare the material that helped with what came next, and the model starts to keep notes. A memory can be placed by position, where it arrived; by content, what it resembles; or into connections, what it taught.

• The memory was non-differentiable. Keys and values were stored as computed and never revisited.
• The memory held the model’s own representations, so its retrieval was a form of attention. Earlier methods fetched text from a corpus or intervened at the output.
• The keys went stale. A long document spans many training steps, so the query that reads a key is computed by newer weights than the ones that wrote it. Normalization evens out magnitude and does not eliminate staleness.

The gains concentrated on rare tokens: names, citations, function names. To name a lemma, the model often looked up its definition from earlier in the proof. That is explicit memory: a fact the weights do not contain, found where it was written down.

The store is no longer beside the context. Nobody is asking what to keep, because everything is kept. The question is what to read, and the cost of answering it became the bottleneck. The indexer still scores every key, so at ten million tokens nearly all of a sparse attention layer’s prefill time goes to indexing. Sparse attention had made reading cheap and left finding expensive.

The keys are projected into a small latent space, normalized, pooled four at a time, and clustered bottom-up into buckets, each with a centroid and a list of key positions. A query scores centroids rather than keys and takes the best buckets. Tree search descends a hierarchy of buckets, so a query scores O(\log T) centroids. The oracle reads (512+4+512)/128{,}000\approx0.8 percent of the keys and keeps 98 percent of full attention’s accuracy: on this task, most of what attention reads, it does not need. LSA reduces the cost of finding the keys, not of storing them. The cache still grows linearly with the context, it is written in one model’s key space, and it ends with the session.

A teacher must know more than the student, and its targets must be ones the student can absorb. Dense attention knows more, because it has looked at every key. Its target is a ranking of clusters, which 64-dimensional projections can represent. The student never has to know what any key means, only where the teacher would look. Telling a model where to look is cheap: a short distillation with nothing else in the model moving, or no training at all. Teaching a model what it read is expensive: fine-tuning had to be fed hundreds of times its source text in generated inferences.

In each line Latent Sparse Attention comes first and Memorilla second.

• Store: the key–value cache; an embedding matrix outside the model.
• Written by: the model’s forward pass; a frozen encoder.
• Bound to one model: yes; the store no, the reader yes.
• Lasts: the session; across sessions.
• Read: sparse, hard selection in every attention layer; a soft read over every document, once, placed ahead of the prompt.
• Fidelity: exact keys and values; a compressed read that “can lose fine-grained factual details”.

LSA strengthens the first rung, the context. Memorilla is a new form of the third rung, harness memory, held in latent vectors instead of text.

Reading everything hurts. Full context trails Memorilla on every benchmark but one, and D-LSA beat full attention on one model at ten million tokens. It is the distraction issue of 2023, and it warns against treating a longer context as a free good. The caveats: Memorilla’s module is fine-tuned on the training splits of all eight evaluation benchmarks, and full context gets nothing comparable. The decoders are small, and only one model family has been tested. The read is measured as flat up to 512 documents, but a soft read over every document must grow with the store eventually, which is the essay’s own inference.

Ten against fifteen, with wide overlapping intervals. The store lasts one rollout and the module is frozen at evaluation, so nothing carries over between games. And training teaches a new environment “without overwriting prior knowledge” because the model never changes, which also means nothing reaches its weights. What the experiment shows is narrower: a trained reader helps a small model use a context it already has. It does not overturn the claim that in-context learning is the best learner on the ladder.

Consolidation pays after n>c/(r-w) uses: here, after 100. Dividing r by 2.87 gives about 3.5, and the break-even moves to about 290 uses. The units are invented, and the prefill, paid once a session, is left out. Where cheaper finding brings no speedup per output token, the bill does not change.

The deflationary answer: “the case for consolidation is the case for amortization”. Paying once in training buys a competence that would otherwise be rebuilt from context at every use, so cheaper reading changes the bill. The substantive answer: consolidation is representation learning, in which gradient descent discovers structure not specified in advance. Retrieval substitutes for it with no important loss for facts about a user, with some loss for procedural patterns, and falls short for taste, “because the taste is the integration”. Read cost does not enter that argument, so cheap reading leaves it where it stood. The deflationary answer is right about explicit memories, and the substantive one about what consolidation builds beyond them.

• By content: the hippocampus keeps an index of the cortical areas an event activated, and the content stays in cortex. LSA’s bucket cache is an index over a store in the same sense.
• By position: prefrontal working memory holds the elements of a sequence in slots according to ordinal position. The key–value cache places content by position, as slots do.
• Into connections: replay transfers the transition structure of hippocampal memories into prefrontal connections. What moves is structure, not episodes.

A language model holds the first two within its context. Nothing writes into its connections during deployment, apart from a recurrent state that is bounded, carries settings and not content, and ends with the session. A context store with a hippocampal index and no replay is a hippocampus without sleep. The disanalogy: the brain’s working memory holds a few items, and that limit is part of what forces consolidation, while the model’s limit is cost. The difference touches only the deflationary case.

A context cache is experience recorded by an older checkpoint and read by the current one. For tokens, a language model’s importance ratios are exact, so a stale rollout can be reweighted. A stale key has no ratio. It is not a sample with a probability under one policy and another. It is a vector computed by a function that no longer exists. The shift is closer to a change of coordinates, and reweighting does not change coordinates. An index adds a second layer: D-LSA’s projections were distilled from a frozen teacher, so a weight update stales the index space as well as the keys. That last step is the essay’s inference.

• Re-encode: run the context again under the new weights. This is the on-policy option, and its cost is a prefill.
• Tolerate: normalize, and let the queries adapt. The evidence comes from a small memory, and normalization fixes magnitude, not direction.
• Translate: learn a map from the old key space to the new. A key at a deep layer depends on everything below it, so correcting stored keys alone is generally insufficient.
• Constrain: change the weights only in ways that keep old keys valid.
• Keep keys the model did not compute: a frozen encoder outside the model writes the store, at the cost of an encoder never optimized for compression. The staleness then moves to the reader.

The only key that cannot go stale is one the model did not compute. A cache in the model’s own key space is exact and fragile. A store outside it is durable and vague.

• The line is one of ideas, not of citations. Latent Sparse Attention cites none of the 2022 and 2023 papers, and the closeness of Quiet-STaR to the note is a convergence, not an influence.
• The brain comparison is the authors’. LSA makes no brain claim, and reading “learning through dynamics” as in-context learning is the essay’s step.
• Three inferences are flagged: that a soft read over every document must grow with the store, that a weight update stales the index space, and that with an outside store the staleness moves to the reader. The last is “still untested”.
• The break-even’s units are invented.

The third placement is open. The essay knows of no deployed frontier model that consolidates what it reads into its weights. The experiment it asks for is a reader, or a model, trained on one store’s history over a long use and compared with the generic one.