Path Patching

Path patching operates by running the model on two distinct inputs: a source prompt and a target prompt. The activation from the source prompt is copied along a specific path—such as from an attention head in layer 10 to the residual stream of layer 15—while all other activations are taken from the target prompt's forward pass. This isolates the direct effect of that single path, answering the counterfactual question: 'How would the model's output change if only this path processed different information?'

Standard activation patching replaces an entire component's output at a single layer, which captures both the direct effect on the final logits and the indirect effect mediated through downstream components. Path patching is more surgical:

• It freezes all downstream mediators by using the target prompt's activations for them
• It isolates only the direct causal pathway between a sender and receiver
• This prevents double-counting effects that flow through multiple routes
• It enables the decomposition of a component's total effect into its constituent paths

The technique is formalized as a three-step intervention on the computational graph:

1. Sender Hook: Capture the activation tensor from the source prompt at the sender component (e.g., an attention head's output)
2. Receiver Hook: During the target prompt's forward pass, replace only the input that the receiver component reads from the sender with the captured source activation
3. Freeze All Else: Ensure every other component's input comes from the target prompt's activations

The difference in logits between this patched run and the clean target run quantifies the path's direct causal contribution.

Path patching is a cornerstone technique in automated circuit discovery and manual reverse engineering:

• Validating hypothesized circuits: After identifying a candidate subgraph, researchers patch through only the proposed path to verify it reproduces the full model's behavior
• Measuring path importance: By systematically patching every path between two layers, researchers rank which routes carry the most causally relevant information
• Decomposing attention head interactions: It reveals how induction heads compose with previous token heads to perform in-context learning
• Isolating MLP contributions: It separates the direct effect of an MLP layer on the residual stream from its effect mediated through subsequent attention layers

Path patching and causal scrubbing are complementary evaluation strategies:

• Path patching is a constructive method: it tests whether a specific path is sufficient to transmit a causal effect by activating only that path with source information
• Causal scrubbing is a destructive method: it tests whether a hypothesized circuit is necessary by corrupting all activations outside the circuit and checking if performance is preserved
• Together, they provide necessity and sufficiency proofs for a circuit's functional role
• Both rely on the same underlying intervention machinery but apply it in opposite directions

Practical path patching requires careful engineering:

• Hook placement: Incorrectly specifying the sender or receiver tensor can contaminate the experiment with unintended causal pathways
• Path granularity: Paths can be defined at the level of individual attention heads, entire layers, or specific weight matrices within an MLP
• Computational cost: Each path requires a separate forward pass, making exhaustive search over all paths expensive for large models
• Activation caching: Efficient implementations cache the source and target activations to avoid redundant computation
• Libraries: Frameworks like TransformerLens and nnsight provide built-in abstractions for defining and executing path patching experiments on models like GPT-2 and Pythia

A circuit is a sparse, interpretable subgraph of connected attention heads and MLP neurons that implements a specific, human-understandable algorithm. Path patching is the primary tool for circuit discovery, as it allows researchers to test whether a hypothesized edge between two components is causally necessary. Key circuit families include:

• Induction heads: Copy patterns from previous occurrences
• Previous token heads: Attend to the immediately prior token
• Name mover heads: Copy entity names in factual recall tasks

A systematic evaluation methodology that tests a hypothesized circuit by replacing all activations outside the circuit with corrupted values and verifying that model performance is preserved. Path patching is used during the iterative refinement of circuits before they are validated with causal scrubbing. If a circuit passes causal scrubbing, it means the identified subgraph is sufficient to explain the behavior—all other components can be ablated without degrading performance on the target task.

A linear decomposition technique that attributes the final logit output for a token directly to the individual contributions of each attention head and MLP layer. While path patching is a causal method requiring multiple forward passes, direct logit attribution is a correlational method computed in a single forward pass using the residual stream's linear structure. The two techniques are complementary: direct logit attribution generates hypotheses, and path patching validates them causally.

The central information highway in a transformer architecture where each layer reads from and writes back to a shared, accumulating state vector. Path patching operates by intercepting the residual stream at specific points—freezing the direct path between a sender component's write to the stream and a receiver component's read from it. Understanding the residual stream's linear, additive structure is essential for designing valid path patching experiments.

Algorithmic methods that automatically identify the minimal subgraph of a neural network responsible for a specific behavior without manual human inspection. These methods—such as ACDC (Automatic Circuit DisCovery) and Subnetwork Probing—use path patching as their core causal scoring mechanism. They systematically test edges between components, using the patching results to greedily construct or prune circuits until a minimal, faithful subgraph is found.