Superposition and Polysemantic Neurons

Interpretability researchers examining individual neurons inside trained networks keep finding the same puzzle: a given neuron activates strongly for several seemingly unrelated things, say French-language text, the numeral 3, and descriptions of cats, consistently, across many inputs, not as noise. This is polysemanticity, and for years it made neuron-by-neuron interpretability look hopeless. Elhage et al., 2022 gave the leading explanation using small toy models with a known ground truth: superposition.

The core argument

Build a toy autoencoder that must represent more input features than it has hidden dimensions. If the features are sparse, meaning each one is only relevant for a small fraction of inputs, and the features carry enough value to be worth representing at all, the network does not simply drop the excess features. Instead, it learns to assign each feature a direction in the hidden space that is only approximately orthogonal to the others, not exactly orthogonal. This creates a small amount of interference between features whenever two happen to be active simultaneously, but because activity is sparse, that interference is rare enough to be a good trade: the network gets to represent far more features than dimensions, at the cost of occasional cross-talk.

This directly explains polysemanticity. A single coordinate, or neuron, that appears to represent several unrelated things is really the readout of several near-orthogonal feature directions overlapping at that coordinate. There is no "French detector neuron" and separate "number 3 detector neuron"; there is one neuron that happens to sit at the intersection of both features' near-orthogonal directions.

Where this connects

Superposition is a direct consequence of the residual stream's fixed width (see the-residual-stream): d_model is fixed, but the number of distinct concepts, facts, and syntactic roles a large model needs to track vastly exceeds it, so the stream is chronically over-subscribed, and packing features via near-orthogonal directions becomes the model's only option. This same crowding is part of why embedding spaces show unexpected geometric structure, including anisotropy (see embedding-space-geometry-anisotropy): a space under pressure to represent more than it has room for does not stay neatly spread out.

The practical response has been sparse-dictionary learning, training a much wider sparse autoencoder on a layer's activations to try to unpack the crowded superposed directions back into individual, monosemantic features, one direction per genuinely distinct concept, at the cost of far more dimensions than the original layer had.

When it falls down

• The theory is validated on toy models with known ground-truth features. Extrapolating cleanly to real transformers trained on messy natural data is well supported by follow-up work but not proven with the same rigor as the toy setting.
• More packed features means more interference under distribution shift. A direction that was clean enough during training can produce unexpected co-activation on out-of-distribution inputs, a plausible contributor to some failure modes, though establishing a direct causal link in any specific case is hard.
• Sparse-autoencoder "features" are themselves an approximation. Dictionary size and sparsity penalty choices change which features get recovered, and there is no independent ground truth in a real model to check the decomposition against.
• Confirming the theory's scaling predictions in production models is difficult. Superposition should increase as a layer gets narrower relative to the number of features it must represent, but you cannot cleanly ablate a trained model's width after the fact to test this directly.