Prompts for Who Teaches the Weights?

Starting the essay

Let’s start writing the essay on continual learning, and continual-learning related RL. On top of what we already explored, see notes/research/ , let’s also cover VPO: see notes/private/VPO-Yacine_podcast.vtt , also https://arxiv.org/html/2605.22817v1 etc. I’m curious if VPO line of research can be connected with Maei and Sutton’s 2010 paper notes/private/GQ_lambda-TD_prediction_with_eligibility_traces-1920.pdf ?

Going deeper through the RL essays

Can we go deeper technically if we connect to the two prior essays on RL (four Substack articles)? notes/how-can-a-reward-train-a-neural-network.md and notes/how-does-a-policy-improve.md

Review

Is it worth reviewing the outline? Otherwise, we can start with Part 1, who-teaches-the-weights.

Unpublished results

Apply all the proposed Part 1 edits. I think asking Lichkovski is not a prerequisite because our blog has a very tiny reach (even relative to Yacine’s podcast).

Length

NB the target length for our blog is 4500 words to 7000 words, 8000 words is still comfortable, 11000 words is a split forcing hard limit.

But we should never aim for a given number of words, this is backwards. We should convey the valuable information concisely and split when it’s not cohering or too much.

Who is the best teacher

“My answer is that the best teacher available is usually the model itself, given better context.” Can you make this paragraph tighter? Is this our insight, or are we channeling the sources? Naively, a stronger teacher is better, and there might be teaching skill/ability separate from object-level abilities.

Fisher information

Maybe worth adding a box explaining Fisher information?

Let’s add a link to a Wikipedia or other source for the generic derivation (KL-divergence -> Fisher information) with related information theoretical or statistical intuitions.

Links

Should we add links to the “classic methods” bullet-points?

Yes, add inline links to papers at first mention across all three parts, like How Does a Policy Improve?. I thought footnotes might be good but the definition-highlight-style links are better.

Let’s expand EWC acronym in the sentence that introduces it

The second reference

“The repair is a second, permanent reference: a frozen copy of the base model, with its own penalty term.” This comes out of the blue, where is there a second reference? Is it part of Mean Teacher?

Evolutionary search

We have an essay touching on evolutionary algorithms.

Editorial pass

Part 1 looks good! Would you like to do an editorial pass and add an entry to the index, or should we wait till we cover the rest first?

Do the editorial pass on Part 1 now; hold the index entry until Parts 2 and 3 are done. Thanks!