Linear Representation Hypothesis

The core claim is that a concept (e.g., 'honesty', 'French text', 'Python code') is represented not by a single neuron, but by a linear direction—a vector—in the model's activation space. This means the model's internal representation of a concept can be mathematically described as a straight line. The intensity of the concept in an input is measured by the dot product between the input's activation vector and the concept's direction vector. This linearity makes concepts amenable to algebraic manipulation.

The primary empirical support comes from probing classifiers. A simple linear model (like logistic regression) trained on a network's internal activations can easily predict a high-level property (e.g., part-of-speech, sentiment). Key implications:

• If a linear probe can decode a concept, the information is linearly separable in the representation space.
• This suggests the network itself is already using a linear encoding, as a non-linear probe would be required to untangle a non-linear representation.
• The probe's learned weights form a vector orthogonal to the decision boundary, which is interpreted as the concept direction.

A powerful consequence is that concepts can be manipulated with simple linear algebra. This is most famously demonstrated with word embeddings (e.g., Word2Vec, GloVe) where vector('King') - vector('Man') + vector('Woman') ≈ vector('Queen'). In transformer models, this extends to activation engineering:

• A steering vector for a concept (e.g., 'refusal') can be derived.
• Adding or subtracting this vector from the model's residual stream during a forward pass directly modulates the model's behavior.
• This allows for controlled generation without any prompt engineering, providing causal evidence for the hypothesis.

The Linear Representation Hypothesis is in direct tension with the phenomenon of superposition. Superposition posits that models compress more features than they have dimensions, leading to polysemantic neurons that fire for multiple unrelated concepts. The resolution is that concepts are still linear directions, but these directions are not axis-aligned (i.e., not a single neuron). Instead, they are sparse, almost-orthogonal vectors in a high-dimensional space. Sparse autoencoders are a key tool used to disentangle these superimposed features into a set of monosemantic, linear directions.

Correlation from probing is not causation. The hypothesis is causally validated using activation patching and direct logit attribution. These techniques confirm that the linear direction is not just readable, but functionally used by the model:

• Activation Patching: Replacing an activation along a hypothesized concept direction with a counterfactual value changes the model's output in a predictable way.
• Direct Logit Attribution: The final model output (logits) can be linearly decomposed into the sum of individual component contributions, showing that the model's computation is fundamentally additive and linear in the residual stream.

The hypothesis is architecturally grounded in the residual stream of transformers. Each layer reads from and writes its output back to this shared, accumulating state vector. This design naturally encourages a linear representation scheme:

• Each attention head and MLP layer adds its output vector to the stream.
• The final representation is a linear sum of all component outputs.
• This additive structure makes it optimal for components to communicate by writing information into specific, independent linear directions, allowing later layers to read them without destructive interference.