Circuits

A valid circuit must be sparse—it includes only the smallest set of model components (attention heads and MLP neurons) necessary to recover the original model's performance on a specific task. Extraneous components are excluded.

• Causal Scrubbing is the primary validation method: all activations outside the hypothesized circuit are replaced with corrupted values.
• If the model's performance is preserved, the circuit is considered complete.
• If performance degrades, the circuit is missing critical components.
• Contrast with the full model, which contains many redundant or task-irrelevant pathways.

Circuits are not flat collections of components; they exhibit hierarchical composition. Earlier layers extract low-level features that are combined by later layers into higher-level abstractions.

• Attention heads route information between token positions, forming skip-trigram patterns.
• MLP layers act as associative memory, amplifying or suppressing features.
• The residual stream serves as the communication channel where each component reads from and writes to a shared state.
• Example: In an Indirect Object Identification circuit, earlier heads duplicate the subject token, and later heads attend to the duplicated position to predict the indirect object.

Every hypothesized circuit must be causally testable. Correlation between a component's activation and a behavior is insufficient; the component must be shown to be necessary and sufficient.

• Knockout analysis: Ablating a circuit component (zero ablation or mean ablation) must degrade task performance.
• Knock-in analysis: Preserving only the circuit components while corrupting all others must restore performance.
• Path Patching isolates the direct effect of a specific computational path between two components, ruling out indirect influences.
• This distinguishes genuine mechanistic understanding from mere activation correlation.

Circuits exhibiting universality reappear with similar structure across different random initializations, architectures, and training runs when models are trained on similar data distributions.

• Induction heads have been found consistently across transformer models of varying scales and architectures.
• Universality suggests circuits are convergent solutions to computational subproblems imposed by the data.
• Not all circuits are universal; some are idiosyncratic to a specific training run.
• Detecting universal circuits provides evidence that the discovered algorithm is a natural solution rather than an artifact of analysis.

Individual neurons in a circuit are often polysemantic—they respond to multiple unrelated concepts. Decomposing these into monosemantic features is critical for interpretability.

• Sparse Autoencoders are trained on activations to produce a sparse overcomplete basis where each latent dimension corresponds to a single concept.
• Dictionary Learning extracts a set of basis vectors that sparsely reconstruct activations.
• In a circuit context, this decomposition reveals that what appears as one neuron may actually be implementing several distinct sub-computations.
• This is essential for understanding circuits exhibiting superposition, where more features than dimensions are compressed into a layer.

A fully understood circuit must be verifiable through end-to-end replication: the hypothesized algorithm, when implemented in code, should reproduce the model's behavior on all inputs.

• Direct Logit Attribution decomposes the final output into additive contributions from each component, confirming the circuit accounts for the total effect.
• The circuit's algorithm should generalize beyond the examples used to discover it.
• Edge cases and failure modes of the circuit should be predictable from the hypothesized mechanism.
• This standard distinguishes partial explanations from complete reverse engineering of the learned algorithm.