Prompts: What Persists When the Agents Change?

These are Łukasz Stafiniak’s substantive prompts that led to “What Persists When the Agents Change? Shared Models, Divergent Inquiries, and the Organization of Learning.” They record the human framing, corrections, and source selection separately from Codex’s research and drafting. Local paths identify the supplied paper versions. A brief wording question and a retry message are omitted.

What do you think of this reaction to a blog Claude wrote with me, at which-agi.md ? https://michalryszardwojcik.substack.com/p/compiled-versus-interpreted-skills Start by reading the synopsis, the full discussion is probably lengthy.

“can carry only what is consistent across the population” – I did let that slide against my intent. This lands painfully close to a very horrible argument that LLMs are midwits because they average over the internet corpus.

The reading of what “population” stands for does a lot of work. If we rather said “can only carry what is consistent given a context” that would be more defensible, but even that, I’m unsure about because of stochasticity. There could be bifurcation points where trajectories are consistent even though endpoints are inconsistent with each other.

Is it worth turning our reaction into an essay for the blog, or would you say a response comment on MRW’s post is enough? If we expand this into an essay, I would maybe cover evidence from multi-agent deployments or happenings.

On our blog, we aim for between 4,000 and 7,000 words, with a hard limit (force-split) around 10,000 to 11,000 words. I think there should also be some research on homogeneous versus heterogeneous agents. I’ll look into it and give you pointers if I find them.

Some options: (1) recent events: Noam Brown interview https://www.dwarkesh.com/p/noam-brown (2) “foundational” article ~/Downloads/factuality_through_debate-arXiv-2305.14325v1/text/ or https://arxiv.org/abs/2305.14325 (3) ~/Downloads/scaling_discovery-arXiv-2609.21032v1/sections/ or https://arxiv.org/abs/2609.21032 (4) ~/Downloads/transactive_memory-arXiv-2606.19911v1/ or https://arxiv.org/abs/2606.19911 (5) ~/Downloads/intrinsic_memory_agents-arXiv-2508.08997v2/ or https://arxiv.org/abs/2508.08997v2

I couldn’t find the heterogeneity benefit remark I vaguely remembered. I think my broader concern about taking an externalist view on intelligence is in agreement with MRW. And I think my tentative narrower concern, where we disagree, about the need for compiled, trajectory-contextualized representations does not survive what we have so far. Let’s also locate this essay in relation to the most recent we wrote: which-world-model.md

Potential source (6) https://www.lesswrong.com/posts/csRby7mZgjL5jCoLL/thoughts-on-the-persona-selection-model

Sounds good. I’m curious how your essay lands. Let’s see!

“They provide working mechanisms for several capacities we had assigned too exclusively to individually compiled experience.” I don’t think this is fair to “Which AGI?” In that essay, we spoke of intelligence cogs, and in the later part we worried about homogeneity of the base models or neural network components, but we never attributed that homogeneity to being a missing cog. My speculative concern there was about reachability of qualitatively different representational spaces when training is vs. is not insulated into trajectories.

I would remove “They elaborate what an interpreted profile can achieve, rather than supplying cogs the earlier essay had declared absent.” – your explanation around it matches what I tried to convey, but the replacement is even more sharply what I questioned originally.

I deleted “Immediate agreement is not always desirable.” filler (restatement)

Could you continue a slight tightening pass? Candidates to remove or shorten: “The sequence matters more to our argument than the final score.” “The order of events is explanatory.” etc. Note I haven’t read the rest article beyond these yet

Will it make sense to introduce sections, or is the article still small enough to flow without hierarchy?

Sounds good, let’s add these headings.

I deleted “Immediate agreement is not always desirable.” filler (restatement)

Will it make sense to introduce sections, or is the article still small enough to flow without hierarchy?

Sounds good, let’s add these headings.

I deleted “Its externalist account and its distinction between filled cogs and the character of their fillers remain in place.” Thank you for writing this! Stepping back, how do you feel about the essay?

News: let’s update the article to include our account of the following reporting: https://thezvi.substack.com/p/ai-187-coming-into-play?open=false#%C2%A7anthropic-approaches-recursive-self-improvement Here I quote the section in full so you don’t need to fetch the long blog post:

[The full section “Anthropic Approaches Recursive Self-Improvement” was supplied as an attachment.]

Would you like to revisit Michal’s original discussion with Thomas Epistemes to see if we represent their position correctly and see if we react to the most valuable points? Just in case, I attach the full transcript that they linked to from the Substack summary.

[The full transcript was supplied as “MRW-compiled-vs-interpreted.txt”.]

Glad you took an in-depth look, go ahead with the updates. Thank you!