Tuned Lens

Unlike the Logit Lens which applies the unembedding matrix directly, the Tuned Lens learns a distinct linear map (a weight matrix) and a bias vector for each layer. This affine transformation corrects for the systematic distortion introduced by interpreting intermediate residual stream states through the final unembedding matrix, which was optimized only for the final layer. The result is a layer-specific decoder that produces sharply more accurate probability distributions over the vocabulary.

The Tuned Lens is trained via a distillation procedure that requires no labeled data. For each layer, a decoder is optimized to minimize the KL divergence between its output distribution and the model's final softmax distribution on a corpus of text. This forces the learned transformation to extract the maximum predictive signal from the intermediate representation, effectively distilling the final layer's knowledge backward through the network.

A core advantage of the Tuned Lens is its causal interpretability. Because it reads from the residual stream without modifying it, the technique does not interfere with the model's forward pass. This allows researchers to trace how a prediction evolves layer by layer without altering the computation itself, unlike activation patching or ablation methods that are inherently interventional.

The original Logit Lens suffers from representation drift: early layers encode information in directions misaligned with the unembedding matrix, yielding noisy or uniform token predictions. The Tuned Lens corrects this by learning a rotation and scaling of the residual stream at each layer. Empirically, the Tuned Lens achieves dramatically lower perplexity on decoded predictions, especially in early and middle layers, revealing that the model converges on its final answer much earlier than the Logit Lens suggests.

By applying the Tuned Lens at every transformer layer, researchers obtain a prediction trajectory showing how the model's top-1 token prediction evolves from input to output. Key observations include:

• Early convergence: Correct answers often dominate by 30-50% of network depth
• Sudden shifts: Abrupt changes in predicted tokens reveal specific layers where critical computation occurs
• Confidence calibration: The trajectory shows how probability mass concentrates on the final answer

The Tuned Lens has notable constraints. It only decodes the linear extractable information from the residual stream; non-linear transformations or information distributed across multiple layers may be invisible. Additionally, the distillation objective biases the decoder toward the final output, potentially overstating early-layer certainty by learning to ignore genuinely ambiguous intermediate states. Researchers must cross-validate findings with causal methods like path patching.

The foundational decoding technique that applies the unembedding matrix directly to intermediate residual stream activations. This reveals the model's next-token predictions at each layer without any learned transformations. While computationally cheap, the logit lens often produces noisy or incoherent outputs in early layers because representations are not optimized to be directly decoded by the final unembedding matrix. The Tuned Lens directly addresses this limitation by learning an affine transformation for each layer.

The central information highway in a transformer architecture where each sublayer reads from and writes its output back to a shared, accumulating state vector. The residual stream acts as a communication channel, allowing attention heads and MLP layers to pass information forward. The Tuned Lens operates by applying a learned transformation to this stream at each layer, decoding the model's evolving internal belief state into interpretable token predictions.

A linear decomposition technique that attributes the final logit output for a token directly to the individual contributions of each attention head and MLP layer. By analyzing the direct path from each component's output through the residual stream to the final logits, researchers can quantify which parts of the network are most responsible for a prediction. This method is closely related to the Tuned Lens, which provides a more accurate per-layer decoding for such attribution analysis.

A simple supervised model, often a linear classifier, trained on a network's internal activations to predict a specific linguistic or world property. This tests whether information is linearly encoded in the representation space. The Tuned Lens can be viewed as a specialized probing classifier that is trained to decode the next token from each layer's residual stream, providing a window into the model's predictive state at every depth.

A causal intervention technique that replaces a model's internal activation at a specific layer and position with a value from a corrupted or alternative forward pass. This isolates the function of a specific component. When combined with the Tuned Lens, researchers can observe how patching an activation changes the decoded predictions at subsequent layers, providing causal evidence for how information flows and transforms through the network.

An unsupervised architecture trained to decompose a model's dense, polysemantic activations into a sparse set of interpretable, monosemantic features. While the Tuned Lens decodes the residual stream into token predictions, sparse autoencoders decompose the same activations into a dictionary of human-understandable concepts. Both techniques aim to make the internal state of the model transparent, but at different levels of abstraction.