Polysemantic Neuron

A polysemantic neuron is a single neuron that activates for a diverse set of unrelated input features. Unlike a monosemantic neuron that fires for one concept (e.g., 'car'), a polysemantic neuron might fire for 'cat faces,' 'car fronts,' and 'cat legs.' This occurs because neural networks exploit the superposition hypothesis, packing more features into a high-dimensional activation space than there are dimensions by using nearly orthogonal directions.

The superposition hypothesis posits that models represent more independent features than they have neurons. Features are encoded as almost-orthogonal vectors in activation space. This allows compression but causes individual neurons to participate in representing multiple features. The phenomenon is most acute when features are sparse—appearing rarely in the data—making it computationally efficient to pack them together rather than dedicating a full neuron to each.

Direct interpretation fails because a single neuron's activation cannot be mapped to one human-understandable concept. When a neuron fires, an observer cannot determine which of its multiple features caused the activation. This ambiguity blocks naive feature visualization and attribution efforts. Researchers must instead use techniques like sparse autoencoders or dictionary learning to disentangle the superimposed representations into a set of monosemantic feature directions.

Sparse autoencoders are the leading tool for decomposing polysemantic neurons. They are trained to reconstruct a layer's activations while enforcing an L1 sparsity penalty on the hidden representation. This forces the autoencoder to learn an overcomplete basis of interpretable, monosemantic features. The original polysemantic neuron's activation is then explained as a sparse linear combination of these learned feature directions, effectively 'unpacking' the superposition.

Monosemanticity is the ideal state where one neuron corresponds to one concept. Achieving it is a central goal of mechanistic interpretability. While some neurons in small models appear monosemantic, scaling laws suggest polysemanticity increases with model size. The field's focus is therefore shifting from finding naturally monosemantic neurons to engineering post-hoc disentanglement methods that impose monosemanticity on the learned feature basis.

In vision models, classic examples include neurons that respond to both cat faces and car fronts—sharing a low-level texture pattern like a curved edge with a central element. In language models, a single neuron might activate for both positive sentiment and academic writing style, two concepts that are statistically correlated in training data. Feature visualization on such neurons produces uninterpretable, mixed imagery, confirming the lack of a single conceptual trigger.

The theory that neural networks represent more independent features than they have dimensions by encoding them in overlapping, nearly orthogonal directions within a shared activation space. This explains why individual neurons appear polysemantic—they participate in representing multiple features simultaneously. The model compresses sparse features into dense vectors, trading off representational capacity for interference between unrelated concepts.

The property of a neuron or feature that activates exclusively for a single, human-interpretable concept. This represents the ideal decomposition goal of mechanistic interpretability. A monosemantic neuron might fire only for 'curly quotation marks in Python code' rather than a mix of syntax, language, and formatting. Achieving monosemanticity requires disentangling superimposed representations through techniques like sparse autoencoders.

An unsupervised neural network trained to reconstruct activations while enforcing sparsity in its hidden layer. Key characteristics:

• Hidden layer has more dimensions than the input
• L1 penalty forces most units to zero
• Decomposes polysemantic neurons into interpretable, monosemantic features
• Learned features often correspond to distinct, human-understandable concepts
• Primary tool for dictionary learning on model internals

A sparse coding approach applied to model activations to find an overcomplete basis of interpretable feature directions. The goal is to decompose the superimposed representations of a neural network into a larger set of nearly orthogonal vectors, each corresponding to a distinct concept. This transforms the problem from 'what does this neuron do?' to 'which dictionary features are active for this input?'

An optimization-based method that generates synthetic inputs to maximally activate a specific neuron, channel, or feature. By starting from random noise and optimizing toward maximum activation, researchers can visualize what pattern the model has learned to detect. For polysemantic neurons, this often reveals a confusing superposition of multiple unrelated visual or conceptual patterns rather than a single coherent feature.

A technique for training a simple linear classifier on frozen internal representations to diagnose what information is encoded at a specific layer. If a linear probe can reliably predict a concept from a layer's activations, that concept is linearly encoded there. This helps identify whether polysemanticity arises from the representation itself or from the probe's inability to disentangle non-linear feature interactions.