Interchange Intervention

An interchange intervention tests a causal hypothesis about a model component. The process involves:

• Base Input: An input where the target variable has one value (e.g., 'The Eiffel Tower is in Rome').
• Source Input: An input where the variable has a different value (e.g., 'The Eiffel Tower is in Paris').
• Intervention: The activation of a specific component (e.g., an MLP layer) from the source input is patched into the model's computation for the base input.

If the model's output flips to match the source input's variable (e.g., predicting 'Paris' instead of 'Rome'), it provides strong evidence that the intervened component causally mediates that specific knowledge.

While often used synonymously, a key distinction exists:

• Activation Patching is the general technique of replacing any internal activation with a cached value from another forward pass. It is a broad tool for localizing a computation.
• Interchange Intervention is a specific experimental design that uses activation patching to test a causal abstraction. It requires a pre-defined hypothesis about a symbolic variable and counterfactual inputs that flip that variable's value.

The interchange intervention is the experimental protocol; activation patching is the surgical tool used to execute it.

Interchange interventions are the primary empirical method for evaluating causal abstraction. The goal is to prove that a neural network's internal components align with the variables in a high-level causal model.

Example Hypothesis: An MLP layer computes the boolean variable 'is_city_a_capital'.

• Base Input: 'The capital of France is London' (variable = False).
• Source Input: 'The capital of France is Paris' (variable = True).
• Intervention: Patch the MLP activation from the source into the base input's forward pass.
• Result: If the model's prediction changes from 'London' to 'Paris', the MLP's activation is causally interchangeable with the 'is_city_a_capital' variable.

A rigorous interchange intervention requires careful dataset construction:

• Counterfactual Pairs: A dataset of paired examples where only the hypothesized variable changes while all other context is held constant.
• Clean Interventions: The patched activation must be the only change to the model's computation. No other state should leak from the source input.
• Component Specificity: The intervention must target a precise location, such as a specific layer's MLP output or an individual attention head's result, to isolate the functional role.
• Metric: Success is measured by the degree to which the model's output behavior shifts toward the source input's target variable, often reported as the interchange intervention accuracy (IIA).

IIA is the standard metric for quantifying the success of an intervention. It measures how well the model's output aligns with the counterfactual target.

• High IIA: Indicates the intervened component is a faithful causal mediator of the hypothesized variable. The model's behavior is fully determined by the patched activation for that specific task.
• Low IIA: Suggests the component does not solely encode the variable, or that the variable's computation is distributed across multiple components.

IIA is often compared against a baseline of a random or zero intervention to establish statistical significance.

Interpreting interchange interventions requires caution due to several failure modes:

• Distributed Representations: If a variable is encoded in a superposed manner across many neurons, intervening on a single component may show a low IIA, leading to a false negative about its involvement.
• Downstream Compensation: Later layers may be robust to the intervention and 'correct' the patched-in information, masking the true causal role of the earlier component.
• Out-of-Distribution Activations: The patched activation, when combined with the base input's residual stream, may create a novel, out-of-distribution state that the model processes unpredictably, violating the intervention's assumptions.

A formal framework for evaluating whether a neural network implements a specific high-level causal algorithm. It works by aligning internal components (neurons, attention heads) with variables in a causal graph and using interchange interventions to test if the model's behavior matches the hypothesized abstraction. If swapping activations between aligned nodes produces the expected output change, the abstraction holds.

A direct causal intervention method that replaces a model's internal activation at a specific location (layer, head, neuron) with a cached activation from a different input. By measuring the resulting change in output, researchers localize where a computation occurs. Key variants include:

• Clean-to-corrupt: patching clean activations into a corrupted forward pass
• Corrupt-to-clean: patching corrupted activations into a clean forward pass
• Resampling patching: using activations from randomly sampled inputs as a baseline

A systematic method for identifying the specific hidden states within a transformer that are causally responsible for recalling a fact. The technique works by corrupting the input (e.g., adding noise to the subject token), then restoring clean activations one layer at a time from a clean forward pass. The layer where restoration recovers the correct output is identified as the causal locus of factual recall. This method revealed that MLP layers in middle transformer blocks are critical for fact retrieval.

A statistical framework adapted from epidemiology for quantifying how much a model's output depends on a specific intermediate representation. It measures the natural indirect effect: the change in output when the mediator (internal activation) is set to the value it would take under a counterfactual input, while holding the direct input fixed. This decomposes a model's reasoning into direct and mediated pathways, revealing which representations transmit causal influence.

A foundational causal technique that removes or zeroes out a model component—such as a neuron, attention head, or entire layer—to measure the resulting drop in performance. Types include:

• Zero ablation: setting activations to zero
• Mean ablation: replacing with the dataset mean activation
• Noise ablation: replacing with random noise
• Resampling ablation: replacing with activations from other inputs The magnitude of performance degradation indicates the component's functional importance.

A general class of techniques that modify a model's internal activations during the forward pass to alter behavior without updating the original weights. Unlike interchange intervention's diagnostic focus, these methods are often used for behavioral control. Examples include:

• Activation addition: adding a pre-computed steering vector to the residual stream
• Contrastive activation addition: using paired positive/negative prompt activations
• DoLa decoding: contrasting logits from different layers to reduce hallucinations