Study notes: The Algebraic Mind Meets the Neural World

Words rest on an arbitrary pairing of sound and meaning (Saussure), which calls for associative memory. Grammar is a discrete combinatorial system (Humboldt), which calls for symbolic computation: a finite algorithm that combines symbols such as “Noun” and “Verb” regardless of their content. Regular and irregular inflection express the same grammatical content, pastness, in forms of the same complexity, single words. They nonetheless show categorically different psychological profiles. Pinker assigns regulars to a rule that suffixes anything labelled Verb, and irregulars to a pattern-associating memory in which similar forms reinforce each other.

Both words are headless. A low-life is a kind of person and not a kind of life. To fly out derives from the noun “a fly”, a fly ball, and not from the verb fly. The pipeline that passes a stored irregular form up from the root is disabled, so the irregular is, in Pinker’s phrase, “trapped in memory”, and the regular rule steps in as the default. The twelve circumstances (rare words, unusual-sounding verbs, headless and rootless derivations, children’s overregularization, the aphasias) share nothing except a failure to retrieve a stored form. In each, irregular inflection suffers and regular inflection applies freely.

German weak -t covers fewer than half of the most common verbs, and the -s plural covers a small minority of nouns. Both behave as defaults in the same circumstances as their English counterparts: unusual-sounding words, denominals, names, eponyms, foreignisms, quotations, children’s overregularizations. So a default does not need a majority. History then reverses the causation. Proto-Germanic had a majority of strong verbs, and the dental suffix applied to borrowings and derived forms, where a default was needed on grammatical grounds. English borrowed heavily from French and Latin, and every borrowing was regular because it was rootless. The majority of regulars is the consequence of default suffixation, not its cause.

• Relations between variables: abstract relationships such as identity or concatenation apply uniformly to all instances of a class, not just to memorized exemplars.
• Structured representations: recursive compositions keep the book on the table apart from the table on the book.
• Individuals versus kinds: Felix and cats are tracked by different mechanisms.

He is not anti-connectionist, and he does not deny that the mind is implemented in neurons. His target is the multilayer perceptron trained by backpropagation. He asks for connectionist architectures that implement symbol manipulation in neural hardware.

It outputs [1110]. The weight update for a connection leaving input node i is proportional to the activation of i times an error signal, so a node that is always 0 never changes its connections: input independence. The updates feeding output node j depend on the error at j and on no other output’s error: output independence. The rightmost output was only ever trained to produce 0, and nothing learned about the other three columns transfers to it. The network cannot discover that all columns are treated alike, so it cannot generalize a universally quantified one-to-one mapping outside the training space. Marcus argues that distributed representations, recurrent architectures, more layers and longer training do not remove the limit.

The structure is the set of bindings {lock/L, able/R}, and \mathbf s=\mathbf{lock}\otimes\mathbf L+\mathbf{able}\otimes\mathbf R. With fillers indexing rows and roles indexing columns,

\mathbf s=\begin{pmatrix}1&0\\0&1\\0&0\end{pmatrix},\qquad \mathbf s\cdot\mathbf R=(0,1,0)=\mathbf{able}.

With \mathbf R=(0.6,0.8), unbinding the left role gives \mathbf s\cdot\mathbf L=\mathbf{lock}\,(\mathbf L\cdot\mathbf L)+\mathbf{able}\,(\mathbf R\cdot\mathbf L)=(1,\,0.6,\,0). The roles are no longer orthogonal, so the other filler leaks in. Exact recovery by the inner product needs orthonormal role vectors. The essay notes that linearly independent roles also suffice if one unbinds with dual vectors.

Continuity: information is encoded and processed in continuously varying real numbers, which gives similarity-based generalization and gradient-based learning. Compositionality: complex information is built from simpler parts by systematic composition, which gives generalization to novel combinations of familiar elements. Symbolic AI could go from lock and -able to lockable but not from lock to fasten, which are different discrete symbols. Neural AI could go from lock to fasten, which are nearby vectors, but not to lockable. Smolensky argues that a hybrid fails because a single encoding often has to be continuously graded and compositionally structured at once. Tensor product representations are his answer: filler/role structure embedded within a continuous vector space.

Hard attention precludes gradient-based learning, and unbounded precision is unrealistic. A fixed-depth Transformer with bounded precision corresponds roughly to constant-depth threshold circuits: it cannot reliably count or recognize all regular languages. The Universal Transformer iterates one layer, but bounded-precision activations in a fixed number of dimensions have finitely many configurations. With unlimited iterations it must revisit a state and enter a cycle, so it is equivalent to a finite automaton. A chain-of-thought Transformer writes intermediate results to its output and re-reads them, so its state space grows with computation time and it recovers Turing completeness with softmax attention and bounded precision. The theme is extensible memory: internal states are bounded, and a growing external store bridges the gap. The essay reads this as Marcus’s distinction between pattern association and symbol manipulation in architectural terms.

Expressive power is what a network can represent given the right weights. Learnability is what it can acquire from finite data with a particular learning algorithm. The completeness results say the right weights exist. Marcus says backpropagation cannot find them from incomplete data, and only for networks with several nodes per variable: he shows that one-node-per-variable networks must represent UQOTOMs. Both are correct. The Neural Turing Machine and the Neural GPU narrow the gap with architectures whose inductive biases make the target computations learnable, by adding structure a plain MLP lacks. The open question is whether some architecture and training regime closes the gap without explicitly provided memory or algorithmic structure. Training independence belongs to specific learning algorithms on specific architectures, not to neural computation in general.

The query encodes the role being sought, as the role vector does in unbinding. Each key advertises the role its position occupies. The query-key dot product is approximate role matching, the value carries the filler, and the weighted sum over values is approximate filler extraction. Both retrieve the content of a structural position regardless of what the content is. The analogy fails in two ways. Unbinding is exact given orthonormal vectors, while softmax weights sum to one and return a blend of fillers. A TPR is a tensor that later processing can inspect, while the attention graph exists only during one layer’s forward pass. That is Smolensky’s criticism and the mark of the first generation: the structure is used within a layer and not encoded in the activation vectors passed on. Second-generation models make filler/role structure explicit in those vectors.

The vocabulary is split into disjoint halves V_1 and V_2. In training V_1 appears only with task class C_1 and V_2 only with C_2; the test flips the pairing. Success requires applying a task’s rule to vocabulary never seen with that task. On sequence-to-sequence tasks (copy or reverse a string, as a task-marker token instructs) both models generalized perfectly. On classification tasks (are two sequences copies, is a token repeated, copy or reversal) both failed unless the training data itself mixed the vocabularies. Gröndahl and Asokan’s explanation is that in the sequence-to-sequence task the input serves as an external memory. The marker sets the order in which positions are looked up, and that addressing pattern is the same whatever vocabulary fills the positions. Classification has one decoding step and no position-by-position readout, and the models learn vocabulary-specific shortcuts.

For Marcus: forced onto purely internal computation, the Transformer fails at vocabulary-general rule application, as the absence of internal variable binding predicts. For Smolensky: where attention can read out position by position, the Transformer fills slots independently of content, functionally equivalent to TPR unbinding and without dedicated symbolic machinery. The essay does not place the distinction between the Transformer and an external memory, because the input sequence is architecturally part of the model. It places it between tasks where attention can address individual positions iteratively and tasks where the whole input must be compressed into a single decision.

• The proof. Training independence is a theorem about the bare backpropagation updates on MLPs with binary inputs and several nodes per variable. The essay grants the proof and narrows its scope: optimizers used in practice couple updates across dimensions, and L2 regularization forces outputs to share hidden units. What survives is that UQOTOM generalization needs some inductive bias.
• Architectural necessity. Marcus correctly lists what a system must be able to do. The essay objects that he slides from “must be able to do X” to “must contain dedicated machinery for X”. Emergent dynamics could supply the function.
• The empirical gap. As of 2001 no connectionist model without symbolic machinery had freely generalized a UQOTOM. The models tested had tens of nodes and hundreds of examples, so absence of evidence was not evidence of absence. Marcus granted in principle that “I cannot possibly rule out alternatives that have not yet been proposed”, and the essay says his later commentary treated the limits as in-principle.

The open question is empirical: can a system without explicit symbolic machinery, trained at sufficient scale on sufficiently rich data, develop the functional equivalent of variable binding and compositional generalization? Marcus says no on principled grounds. Smolensky says the mathematics allows it. The Gröndahl–Asokan results suggest the answer depends on whether attention can be used for content-independent structural processing, which sequence-to-sequence tasks permit and classification tasks do not. The essay surveys the debate before large language models and leaves the test at scale to a later article. It also defers two suggestions. CLARION’s two levels might be vindicated as functional organization, with both implemented subsymbolically, if a model with chain-of-thought is read as an implicit substrate producing explicit traces. And attention heads might parallel the slots of working memory, with a possible link to Merge that it calls “an open and tantalizing question”.