Circuit Analysis

A valid circuit must satisfy the minimality criterion: it contains no redundant components. This is verified through ablations—if removing a node does not degrade the target behavior, it is not part of the circuit. Researchers systematically prune the computational graph by zero-ablating attention heads and MLP neurons, then measuring the change in output. The minimal subgraph that preserves performance is the hypothesized circuit. This distinguishes true functional components from correlated but causally irrelevant activations.

Circuit analysis relies on causal interventions rather than correlational methods. Key techniques include:

• Activation Patching: Replacing a model's internal activation at a specific layer and token position with a cached activation from a different forward pass to isolate functional pathways
• Knockout Analysis: Zero-ablating specific attention heads or neurons and measuring the degradation in task performance
• Path Patching: A refined technique that routes activations along specific paths between two layers, isolating the direct effect of one component on another without confounding influences from intermediate layers

Beyond necessity, a circuit must be sufficient to reproduce the behavior in isolation. Researchers test this by freezing all model components outside the hypothesized circuit to their mean activations or by running the model with only the circuit's nodes active. If the circuit alone can replicate the full model's performance on the target task, it passes the sufficiency test. This is formalized in Causal Scrubbing, a hypothesis-testing framework that systematically resamples activations to verify if the proposed circuit faithfully explains the model's behavior under all relevant input distributions.

Circuits are decomposed into functional modules that map to interpretable algorithmic primitives:

• Induction Heads: Attention heads that perform in-context copying by attending to the token following a previous occurrence of the current token, implementing a primitive lookup table
• Previous Token Heads: Heads that simply attend to the immediately preceding token, acting as information conduits
• Suppression Heads: Heads that actively reduce the probability of specific tokens, implementing inhibitory control
• Name Mover Heads: Heads that copy entity attributes from source to target positions in factual recall tasks Each module's role is validated through targeted ablation and activation patching experiments.

Attention heads are analyzed by decomposing them into two independent computational pathways:

• QK Circuit: The query-key interaction determines which previous tokens the head attends to. The QK circuit computes attention scores using the query and key projection matrices, with positional encodings like Rotary Position Embedding (RoPE) ensuring relative distance sensitivity
• OV Circuit: The output-value interaction determines what information is copied from the attended token. The OV circuit transforms the value vectors through the output projection matrix This decomposition reveals that a single head can simultaneously implement a copying mechanism (OV) and a content-based routing mechanism (QK).

Circuit analysis must contend with the Superposition Hypothesis: neural networks represent more independent features than they have dimensions by encoding them in nearly-orthogonal directions. This means individual neurons are often polysemantic—activating for multiple unrelated concepts. To address this, researchers apply Sparse Autoencoders (SAEs) to decompose dense activations into a sparse set of monosemantic features before circuit analysis. This dictionary learning approach reveals that what appears as a single neuron may actually be a superposition of 10-50 distinct features, each participating in different circuits.

A causal intervention technique for isolating circuits. It works by running the model on a 'clean' prompt, caching a specific internal activation, and then replaying it during a forward pass on a 'corrupted' prompt. If performance is restored, the patched component is part of the causal circuit for that behavior. This helps distinguish correlational from causally necessary components.

A formal hypothesis-testing framework that validates proposed circuits. It systematically replaces activations from a 'clean' run with activations from 'corrupted' runs where the circuit's hypothesized function is broken. If the circuit hypothesis is correct, the model's performance should drop to the corrupted baseline. It ensures a circuit is not just sufficient but also faithful to the model's actual computation.

Used to decompose polysemantic neurons into monosemantic features. An SAE is an overcomplete basis trained to reconstruct activations with a sparsity penalty. This forces the model to represent concepts as distinct, interpretable directions. Key variants include:

• Gated SAEs: Separate feature activation from magnitude estimation.
• Cross-Layer Transcoders: Translate features between adjacent layers.

A refined causal intervention that isolates specific computational paths between two components (e.g., a specific attention head's output to a specific MLP neuron). Unlike standard activation patching, it replaces only the direct connection, leaving other downstream paths intact. This is essential for dissecting QK circuits (attention patterns) and OV circuits (information transfer) within attention heads.

A direct method to interpret the residual stream. The Logit Lens applies the final unembedding matrix to intermediate layer outputs to see the model's current 'best guess' for the next token. More generally, linear probes are small classifiers trained on frozen internal activations to detect encoded concepts like syntax, sentiment, or truthfulness, revealing the emergent structure of the model's representations.

The core technique for locating knowledge neurons in MLP layers. It introduces noise to input embeddings and then iteratively restores clean hidden states at specific layers. By measuring which restoration steps recover the correct output, it pinpoints the exact layer and token position where a fact is recalled. This was foundational for Rank-One Model Editing (ROME).