Residual Stream

The residual stream is not a single static vector but a cumulative sum of outputs from all previous layers. Each transformer block adds its output to the stream via a residual connection. Mathematically, the state at layer l is x_l = x_0 + Σ a_i + Σ m_i, where a_i are attention outputs and m_i are MLP outputs. This means the final representation contains a linear superposition of information from every layer, allowing later layers direct access to early token-level features without compression through intermediate bottlenecks.

Residual connections create a gradient superhighway that mitigates the vanishing gradient problem in deep networks. By providing an identity path (y = F(x) + x), the model can learn small, incremental perturbations to the stream rather than full transformations. If a layer learns nothing useful, it can default to near-zero output, effectively skipping the layer and passing information through unchanged. This is why transformers scale to hundreds of layers without degradation.

Attention heads do not directly modify token representations. Instead, they write their results into the residual stream, and MLP layers read from the stream to process information. The residual stream acts as a shared memory bandwidth where heads communicate by writing vectors that other heads in later layers can attend to. This is the basis of induction heads: an early head writes a token into the stream, and a later head reads from that position via the attention pattern, enabling in-context copying.

Features encoded in the residual stream are often linearly accessible. This means a simple linear probe (a logistic regression classifier) trained on the stream's activations can decode high-level concepts like sentiment, syntax, or truthfulness. The Logit Lens technique exploits this by applying the unembedding matrix directly to intermediate residual states, revealing the model's evolving next-token predictions at every layer. This linearity is a core assumption behind representation engineering and steering vectors.

While the residual stream enables powerful computation, it also forces the model to compress vast amounts of information into a single fixed-width vector. This leads to the superposition hypothesis: the model represents more independent features than it has dimensions by encoding them in almost-orthogonal directions. A single neuron or dimension rarely corresponds to one clean concept. Instead, features are polysemantic, activating for multiple unrelated inputs. Sparse autoencoders (SAEs) are used to decompose this dense, entangled stream into a sparse set of monosemantic features.

Mechanistic interpretability treats the residual stream as a computational scratchpad. Subspaces of the stream correspond to specific variables in an algorithm the model has learned. For example, in a modular addition circuit, specific dimensions store the carry bit and the sum. Activation patching intervenes on these subspaces to causally verify their function. By reading and writing to orthogonal subspaces, the model performs complex, multi-step reasoning without interference between parallel computations.

A computational subunit that reads from the residual stream and writes a weighted sum of value vectors back to it. Each head computes attention scores via the QK circuit to determine where to pull information from, and transforms it via the OV circuit to determine what to write. The result is added into the residual stream at the head's output position, enabling information movement between token positions.

A position-wise feed-forward network that reads each token's residual stream state independently and writes a transformed vector back. MLPs act as key-value memories: the first linear layer projects the input to a high-dimensional space where knowledge neurons activate, and the second layer maps the result back down. This is where the majority of factual knowledge is stored and retrieved.

A direct probing technique that applies the unembedding matrix to intermediate residual stream states at any layer. This reveals the model's evolving next-token predictions before the final layer norm. By decoding the residual stream at each layer, researchers can observe how the model gradually refines its output distribution, often finding that the final prediction is largely determined by middle layers.

An unsupervised decomposition technique that breaks the dense, polysemantic activations in the residual stream into a sparse set of monosemantic features. An SAE learns an overcomplete basis where each feature corresponds to a single interpretable concept. This addresses the superposition hypothesis—the theory that models pack more features than dimensions into the residual stream using near-orthogonal directions.

A causal intervention technique that replaces a specific activation in the residual stream at a given layer and token position with a cached activation from a different forward pass. By patching clean activations into corrupted runs (or vice versa), researchers isolate the causal contribution of specific components. This is the primary method for discovering circuits—minimal subgraphs responsible for specific behaviors.

A top-down control method that manipulates high-level representations directly in the residual stream using steering vectors. Rather than modifying individual weights, a direction vector corresponding to a desired behavior (e.g., honesty, refusal) is added to the residual stream at inference time. This provides a lightweight alternative to fine-tuning for controlling model behavior.