Causal Scrubbing

Causal scrubbing operates on a simple principle: if a hypothesized circuit is correct, replacing all other activations with noise should not degrade performance. The method resamples activations from a 'corrupted' distribution where the key causal factors are broken, but only for nodes outside the proposed circuit. If the model's loss on the original task is recovered when the circuit's activations are patched in, the hypothesis is validated. This directly tests the completeness of the proposed subgraph.

The technique relies on activation patching across multiple forward passes:

• Clean Pass: Run the model on the original input to capture the 'correct' activations for the hypothesized circuit nodes.
• Corrupted Pass: Run the model on a perturbed input (e.g., shuffled tokens, noise) to capture 'incorrect' activations for all other nodes.
• Patched Pass: Run a third pass where the circuit nodes use the clean activations, and all other nodes use the corrupted activations. The model's output is then evaluated.

Causal scrubbing is fundamentally different from ablation (zeroing or mean-replacing activations):

• Ablation removes information, creating out-of-distribution states that can cause unpredictable model failures.
• Causal Scrubbing replaces information with a valid, in-distribution alternative from a corrupted dataset. This preserves the model's typical activation statistics while breaking the specific causal relationship under investigation, providing a more faithful test of the circuit's role.

The process is inherently iterative and serves as a debugging tool for mechanistic interpretability:

1. Propose: An initial circuit is hypothesized based on attention patterns or weight analysis.
2. Scrub: The scrubbing test is applied. If performance drops significantly, the circuit is incomplete.
3. Revise: The researcher analyzes the nodes where the patched activations failed to recover performance, identifying missing components (e.g., a forgotten attention head or MLP neuron).
4. Repeat: The expanded circuit is tested again until the scrubbing test is passed, confirming the hypothesis.

A successful causal scrubbing test provides strong evidence that the identified subgraph is a faithful causal model of the behavior. The criterion is formalized as: for a given behavior B and a hypothesized circuit C, the model's performance on B should be statistically indistinguishable when running the full model versus running the model where only C receives clean activations and all other nodes receive activations from a distribution that breaks B. This moves interpretability from correlational observation to interventional proof.

Causal scrubbing integrates concepts from several interpretability techniques:

• Activation Patching: It is a systematic, hypothesis-driven application of patching, rather than an exploratory one.
• Causal Mediation Analysis: It directly tests for a perfect mediation condition, where the indirect effect through the circuit fully explains the total effect.
• Mechanistic Interpretability: It serves as the gold-standard end-to-end validation step after discovering circuits via other means like sparse autoencoders or probing. It answers the question: 'Have we found the entire algorithm, or just a part of it?'

A circuit is the minimal subgraph of a neural network—comprising specific attention heads and MLP layers—that mediates a particular behavior. Causal scrubbing is the primary tool for verifying that a hypothesized circuit is both faithful (captures all relevant computation) and complete (requires no other components). A validated circuit explains the full mechanistic pathway from input to output.

A causal intervention technique that replaces the activations of a specific model component with activations from a corrupted or counterfactual input. By measuring the resulting change in output, researchers isolate which components are necessary and sufficient for a behavior. Causal scrubbing generalizes this by systematically patching entire hypothesized circuits simultaneously to test for completeness.

Quantitative measures that evaluate how well an explanation captures the model's true computational process. A faithful circuit produces identical behavior when run in isolation. Key metrics include:

• Recovery rate: How much of the original performance is restored by the isolated circuit
• Completeness score: Whether the circuit accounts for all relevant information flow
• Minimality: Absence of redundant or non-contributing components

The process of identifying unexpected behaviors in neural networks by analyzing their internal computations. When causal scrubbing reveals that a circuit fails to fully mediate a behavior, the residual error indicates the presence of additional, unidentified mechanisms. This drives iterative refinement of the circuit hypothesis and uncovers backup circuits or polysemantic neurons that encode multiple features.

The theory that neural networks represent more features than they have dimensions by encoding them in almost-orthogonal directions in activation space. Causal scrubbing must account for superposition because a single neuron may participate in multiple circuits. Scrubbing a neuron for one behavior may inadvertently corrupt another, requiring careful feature disentanglement to isolate truly independent mechanisms.

A refined causal intervention technique that isolates specific computational paths between components rather than entire components. While standard activation patching replaces all outputs from a head, path patching targets only the influence flowing from one specific component to another. This enables causal scrubbing at finer granularity, testing whether a particular attention edge or MLP connection is necessary.