Causal Scrubbing

Causal Scrubbing operates on a simple principle: if a hypothesized circuit truly captures the model's algorithm for a behavior, then all activations outside that circuit should be irrelevant. The method systematically replaces activations not in the circuit with values from corrupted inputs—such as noise or shuffled data—and checks if the model's output remains correct. If performance degrades, the circuit is incomplete or incorrect.

Unlike simple zero or mean ablation, Causal Scrubbing uses resampling ablations that replace activations with values drawn from a reference distribution. This distribution is generated by running the model on corrupted inputs where the target behavior is impossible. Key properties:

• Preserves the natural statistics of activations
• Avoids out-of-distribution artifacts that zero ablation introduces
• Provides a more rigorous test than mean imputation

Causal Scrubbing establishes a tree-structured mapping between nodes in the clean computational graph and nodes in the corrupted graph. This correspondence tree specifies which activations get replaced and which are preserved. The tree branches at each layer, allowing fine-grained control over which computational paths are scrubbed versus retained, enabling precise isolation of circuit components.

The output of Causal Scrubbing is a quantitative faithfulness score that measures how well the hypothesized circuit explains the model's behavior. A score of 1.0 indicates perfect preservation of performance under scrubbing, meaning the circuit fully captures the algorithm. Lower scores reveal gaps in understanding. This metric enables:

• Comparing competing circuit hypotheses
• Tracking progress in reverse engineering
• Identifying which behaviors remain unexplained

While Activation Patching tests individual components by replacing single activations, Causal Scrubbing evaluates entire circuits simultaneously. Activation Patching answers 'Is this component necessary?' whereas Causal Scrubbing answers 'Is this entire circuit sufficient?' The two methods are complementary: patching identifies candidate components, and scrubbing validates the complete hypothesized algorithm.

Causal Scrubbing drives an iterative scientific process for mechanistic interpretability:

1. Hypothesize a circuit based on activation analysis
2. Scrub all activations outside the circuit
3. Measure the faithfulness score
4. Identify where performance drops indicate missing components
5. Refine the circuit and repeat This loop transforms circuit discovery from speculation into rigorous, falsifiable science.

The fundamental object of study that causal scrubbing aims to validate. A circuit is a sparse subgraph of a neural network—a specific set of connected attention heads and MLP neurons—that implements a human-understandable algorithm. Causal scrubbing tests whether a hypothesized circuit fully explains a behavior by corrupting all activations outside the proposed subgraph and verifying that performance is preserved.

A core causal intervention technique used as a building block within the scrubbing process. Activation patching replaces a model's internal activation at a specific layer and token position with a value from a corrupted forward pass (e.g., one with shuffled input). This isolates the causal contribution of that specific component. Causal scrubbing systematically applies this idea across all nodes outside the hypothesized circuit.

The specific corruption method at the heart of causal scrubbing. Instead of zeroing out or adding noise to activations, resample ablation replaces an activation with a value drawn from a different, randomly selected input from the dataset. This preserves the marginal distribution of activations while destroying the specific information content. The scrubbed model should fail if the hypothesized circuit is incomplete.

The metric by which causal scrubbing judges a hypothesized circuit. A circuit is faithful if, when all activations outside it are resample-ablated, the model's performance on the target task is statistically indistinguishable from its original performance. Key metrics include:

• Recovery rate: Percentage of original performance retained
• Selectivity: Performance drop when the circuit itself is ablated
• Completeness: Whether the circuit accounts for all task-relevant computation

A more surgical causal method that isolates the direct effect of a specific computational path between two components (e.g., from an attention head in layer 1 to an MLP in layer 3). While causal scrubbing tests an entire circuit simultaneously, path patching decomposes the circuit into individual edges. The two techniques are complementary: path patching helps discover circuits, while causal scrubbing validates them holistically.

Algorithmic methods that automatically propose candidate circuits for causal scrubbing to validate. These techniques—such as ACDC (Automatic Circuit DisCovery) and Subnetwork Probing—use gradient-based search or discrete optimization to identify minimal subgraphs responsible for a behavior. Causal scrubbing serves as the rigorous final validation step, ensuring the discovered circuit is not just correlated but truly causal.