Polysemantic Neuron

A polysemantic neuron is a single artificial neuron that activates for a diverse set of unrelated input patterns or concepts. Unlike a monosemantic neuron, which fires for a single, human-interpretable feature like 'car' or 'French text', a polysemantic neuron might respond to academic citations, DNA sequences, and computer code simultaneously. This makes direct labeling of its function impossible without further decomposition.

Polysemanticity is explained by the Superposition Hypothesis. Neural networks represent more independent features than they have dimensions in a given activation space. They achieve this by encoding features in almost-orthogonal directions. A single neuron can participate in representing multiple features by exploiting high-dimensional geometry, compressing an overcomplete feature basis into a lower-dimensional layer.

The primary tool for resolving polysemanticity is the Sparse Autoencoder (SAE). An SAE is trained to reconstruct a model's dense activations into a higher-dimensional, sparse latent space. This process disentangles the mixed features, producing a set of monosemantic features that are individually interpretable. The SAE effectively performs dictionary learning on the model's internal representations.

To verify if a polysemantic neuron's mixed responses are causally relevant, researchers use Causal Mediation Analysis. This involves patching the neuron's activation during a forward pass and measuring the change in the model's output. If patching the neuron alters the model's performance on tasks related to one of its detected concepts, it confirms the neuron plays a functional role in representing that specific feature.

Polysemanticity is a fundamental obstacle in Circuit Analysis. When tracing the computational graph for a specific behavior, a polysemantic neuron appears to be part of multiple unrelated circuits. This makes it difficult to isolate the minimal subgraph responsible for a single task. Decomposing neurons into monosemantic features is a necessary prerequisite for identifying clean, faithful circuits.

The theory that neural networks represent more independent features than they have dimensions by encoding them in almost-orthogonal directions. This is the primary mechanism believed to cause polysemanticity, as a single neuron can participate in representing multiple compressed features simultaneously. The model leverages the high-dimensionality of activation space to cram features into a lower-dimensional representation without catastrophic interference.

An unsupervised technique used to decompose a model's dense, polysemantic internal activations into a sparse set of interpretable, monosemantic features. By training an overcomplete basis with an L1 penalty, SAEs isolate the independent ground-truth features that were previously superimposed inside a single polysemantic neuron, enabling direct human inspection of model state.

A decomposition method that learns an overcomplete basis of vectors to represent activations as a sparse linear combination of interpretable features. This is the mathematical framework underpinning SAEs. It posits that a polysemantic neuron's activation is not a single concept but a sum of many independent dictionary elements, each corresponding to a distinct, monosemantic feature.

The conceptual opposite of a polysemantic neuron. A monosemantic neuron reliably activates for a single, human-interpretable concept regardless of context. A core goal of mechanistic interpretability is to transform polysemantic representations into monosemantic ones, often by finding the correct sparse basis via dictionary learning rather than expecting native weights to be clean.

A specific neuron within an MLP layer identified through causal tracing that is primarily responsible for expressing a particular piece of factual knowledge. While often treated as monosemantic, many knowledge neurons are polysemantic, activating for multiple related facts or linguistic contexts. Causal mediation analysis is required to verify if the neuron's polysemantic nature affects downstream fact retrieval.

A causal intervention technique that replaces a model's internal activation with a cached activation from a different forward pass. This is used to isolate the function of a polysemantic neuron by patching its value and observing the change in output. If a neuron is polysemantic, patching it to fix one behavior may inadvertently corrupt another, revealing its entangled role in the computational graph.