Polysemanticity

The hypothesized mechanism that causes polysemanticity. A neural network represents more independent features than it has dimensions by compressing sparse, high-dimensional concepts into a lower-dimensional vector space. This allows models to store vast amounts of knowledge but results in individual neurons firing for multiple, seemingly unrelated inputs. The compression exploits the fact that features are sparse—most are inactive at any given time.

The ideal property where a single neuron or feature direction corresponds to exactly one natural, human-interpretable concept. A monosemantic neuron fires exclusively for a specific entity, pattern, or attribute. Achieving monosemantic representations is a primary goal of mechanistic interpretability, as it would make model internals directly auditable. Current models exhibit pervasive polysemanticity, making monosemanticity a target for techniques like dictionary learning.

An unsupervised architecture trained to decompose dense, polysemantic activations into a sparse set of interpretable features. It learns an overcomplete basis of feature directions, enforcing sparsity via an L1 penalty on the latent activations. The result is a set of monosemantic features that collectively explain the model's internal state. Key components include:

• Encoder: Maps dense activations to sparse latent features
• Decoder: Reconstructs original activations from sparse features
• Sparsity constraint: Forces most features to be inactive for any given input

A method for decomposing a model's activations into a sparse linear combination of learned basis vectors, each representing a distinct, interpretable feature. The technique treats the activation space as a dictionary where each basis vector is a 'word' in the model's internal vocabulary. By enforcing sparsity, dictionary learning forces the decomposition to use the minimal set of features needed to explain an activation, naturally separating polysemantic neurons into their constituent monosemantic components.

An optimization-based technique that synthesizes input examples that maximally activate a specific neuron or feature direction. For a polysemantic neuron, feature visualization often reveals multiple distinct patterns corresponding to its different semantic meanings. This technique provides visual evidence of polysemanticity by showing that a single neuron responds to diverse, unrelated concepts. Methods include:

• Activation maximization: Gradient ascent on input to maximize neuron firing
• Dataset examples: Retrieving training samples that cause highest activation

The conjecture that high-level concepts are encoded as linear directions in a neural network's activation space. Under this hypothesis, a concept like 'honesty' or 'French' corresponds to a specific vector direction. Polysemanticity complicates this picture because a single neuron may participate in multiple concept directions simultaneously. The hypothesis motivates techniques like probing classifiers and activation steering, which assume linear separability of encoded knowledge.