Cross-Layer Transcoding

A transcoder is a learned linear or non-linear map that translates the sparse feature activations from a source layer into the sparse feature representation of a target layer. Unlike a standard SAE, which decomposes a single layer, a transcoder explicitly models the computational transformation between layers. It takes a vector of feature coefficients from layer L and predicts the corresponding feature coefficients in layer L+1, revealing how the model refines, combines, or discards features during forward propagation.

A single neuron in an early layer may respond to multiple unrelated concepts (polysemanticity). Cross-layer transcoding demonstrates how these entangled representations are progressively disentangled by subsequent layers:

• An early feature might activate for both 'legal documents' and 'scientific papers'
• The transcoder shows this feature splitting into two distinct, monosemantic features in the next layer
• This provides direct evidence for the superposition hypothesis and how models build hierarchical abstractions

Transcoders enable researchers to verify if a circuit identified in one layer remains functionally intact in subsequent layers. By translating the feature activations of a circuit component forward, one can check if the downstream representation still encodes the same logical operation. This addresses a key limitation of single-layer circuit analysis, where connections between components in different layers are often assumed rather than empirically validated through causal mediation analysis.

A cross-layer transcoder is trained by:

• Collecting paired activations: Run the model on a diverse corpus and cache the sparse feature coefficients from the SAEs of two adjacent layers
• Minimizing reconstruction error: Train the transcoder to predict the target layer's feature coefficients from the source layer's coefficients, typically using an L2 loss or a KL divergence if features are probabilistic
• Sparsity constraint: Apply an L1 penalty on the transcoder's output to ensure the predicted feature set remains sparse and interpretable, preventing the model from activating spurious features

Transcoders provide a systematic framework for tracking the lifecycle of a feature across the model's depth:

• Feature birth: A transcoder identifies when a new, distinct feature emerges in a layer that had no direct counterpart in the previous layer
• Feature death: A feature present in layer L may have near-zero weight in the transcoder's prediction for layer L+1, indicating it has been pruned or fully merged into a higher-level abstraction
• This allows researchers to build a computational phylogeny of concepts within the model

Cross-layer transcoding complements activation patching by providing a predictive model of how features transform. While patching tests causality by replacing an activation and observing the output change, a transcoder predicts the expected downstream feature state. Discrepancies between the transcoder's prediction and the actual patched result can reveal non-linear interactions or the influence of attention mechanisms that route information between token positions, which a purely feed-forward transcoder might miss.

The theory that neural networks represent more independent features than dimensions by encoding them in almost-orthogonal directions. This explains why cross-layer transcoding is necessary.

• Features are compressed into Johnson-Lindenstrauss style embeddings
• Individual neurons become polysemantic, responding to multiple unrelated concepts
• Cross-layer transcoders help track how these compressed feature representations evolve as they propagate through successive layers

The broader mathematical framework for decomposing activations into a sparse combination of learned basis vectors. Cross-layer transcoding applies dictionary learning across the temporal dimension of computation.

• Standard dictionary learning: x ≈ D · c where c is sparse
• Cross-layer variant learns a mapping: c_{L+1} ≈ T(c_L)
• The transcoder matrix T reveals how features transform, combine, or split between layers

A causal intervention technique used to validate transcoder mappings. By replacing activations at specific layers and token positions, researchers can verify whether a transcoder's predicted feature transformation actually occurs.

• Clean vs. corrupted forward passes isolate causal effects
• Patching at the feature level (using transcoder reconstructions) tests if specific feature combinations are necessary
• Provides ground-truth validation that a transcoder has faithfully captured the computational structure