Prompts for Every Layer a Store

Scope

Let’s start working toward the “LLM architectures and training” essay. What’s the lay of the resources we have collected?

About the scope and shape to be covered in this session: my initial thinking was to have a deep account of architectures and training first, probably as one essay for the blog that might land as parts on Substack if it ends up being long; but could split into architectures article and training article if that ends up more natural. I would keep interpretability in a subservient role to understanding the surveys’ content. Where I see the blog going after what we’re discussing in this session: (a) re-evaluating and deepening our account of mentality / mindedness. (b) This is tempting but not fully decided: Responding to critics of AI mindedness (phrased as “proper intelligence”) in the current frontier AIs, coming from the “Partially Examined Life” podcast, I haven’t listened to it yet; engaging with an upcoming video by Richard Brown on consciousness in current AIs (from the brief description the take seems to be “illusionism turns out to be true of AIs”). (c) Clarifying and defending our core metaphysical commitment to ontic structural realism. The topics that are still unclear whether they receive a dedicated essay in near future: a separate interpretability article; a separate article about personhood.

Title

Are the 7 checkpoints distinct enough to merit the title “Seven Ways to Extend the Transformer, and How They Are Trained”? If yes then it’s a good title, otherwise “Every Layer a Store: How the 2026 Open Models Are Built and Trained” is good.

“5 ideas” might also make a good title, WDYT? Otherwise we stick with the current pick.

Sources

Would this survey be a good additional source? Not sure because at first glance it looks a bit armchair’y. https://arxiv.org/html/2504.15965v2

Opening and the store vocabulary

I agree with the suggested opening – open strong with showing how a naive vanilla Transformer view is outdated, the preserve the striking example for later.

Q2. A full section; I’m expecting I’ll ask for the classical lineage background as subsections or boxes.

Kanerva and Hawkins

Is Kanerva’s work worth bringing up? Is it at tension with the recent “NNs are clean” message?

That’s exciting! Agreed, add a note somewhere interpretability-attached (I like the homonymy point). What about Hawkins “Hierarchical Temporal Memory”, is that research direction relevant to the current state-of-the-art?

Ah ok, that’s not worth bringing up then, I’d leave it out of sources of the mindedness essay. At some point we can write about non-gradient / zeroth-order methods, but HTM doesn’t look like a productive candidate to seed that essay.

Structure and figures

I’d keep section 6 in Part A, it fits as architecture consideration.

Q4: is SVG better than Mermaid? Or would it be Mermaid exported to SVG?

Drafting

Okay, ready to start drafting the essay(s)?

Maybe we should already decide to split into essay A and essay B? Draft A in one go.

MAI-Thinking-1

Let’s continue working on the LLMs two-parter, with notes/every-layer-a-store.md covering the architecture and the next essay covering training. Is it worth extending the survey to include MAI-Thinking-1 ? https://microsoft.ai/pdf/mai-thinking-1.pdf

Yes, add to outline (the MAI-Thinking-1 points in the Part B sections, plus the one-sentence aside for essay A).

Fact check

What should I look into next to improve “Every Layer a Store”?

Yes, fact check first (results to a numbers-check note in notes/research/).

Stores table

Yes, stores table (one row per store, columns for the four questions plus how it forgets).

Figures

Yes, figures (Figures 1–3 as matplotlib SVGs with their scripts in the repo).

Editorial pass

Yes, editorial pass (AGENTS.md style).