Residual Stream

The residual stream operates on a simple principle: each layer reads from the stream and writes its output back to it via addition. This means the stream's state at layer l is the sum of the initial token embedding and all previous layer outputs. This additive structure prevents vanishing gradients and allows information to flow directly from early layers to later ones, creating a residual pathway that is crucial for training deep networks.

Because the residual stream is the sole channel for information transfer between layers, it is the ideal target for probing classifiers. A linear probe trained on the stream's state at a specific layer can diagnose what information is encoded at that point:

• Early layers: Encode shallow syntactic features (part-of-speech, morphology).
• Middle layers: Encode semantic roles and local context.
• Later layers: Encode high-level, task-specific abstractions and factual knowledge.

A powerful technique for decoding the residual stream is the Logit Lens. By applying the model's final unembedding matrix directly to the residual stream at an intermediate layer, you can see what the model would predict if it stopped there. This reveals the model's iterative refinement process, showing how its top prediction evolves from generic continuations to the final, confident answer.

The Superposition Hypothesis posits that the residual stream represents more features than it has dimensions. It does this by encoding features in nearly orthogonal directions within its high-dimensional space. This allows a single model to store vast amounts of knowledge, but it leads to polysemantic neurons that fire for multiple unrelated concepts, complicating direct interpretation.

The additive nature of the residual stream makes it a direct interface for controlling model behavior. A steering vector—a direction in the activation space derived from contrasting positive and negative prompts—can be added to the stream at a specific layer during inference. This activation addition technique can reliably induce high-level behaviors like truthfulness, a specific sentiment, or a refusal to answer, without any retraining.

To prove a component's causal role, researchers perform interchange interventions on the residual stream. By replacing a specific activation from a corrupted input with a clean, cached activation from a different forward pass, they can localize a computation. Causal tracing uses this to pinpoint the exact hidden states responsible for recalling a specific fact, confirming the stream's role as the carrier of causally significant information.