Automated Circuit Discovery

The central metric for ACD is causal fidelity—how well the discovered circuit's output matches the full model's behavior when all other components are ablated. A perfect circuit with 100% fidelity would produce identical logits to the original model on the target task. Evaluation methods include:

• Minimality vs. completeness trade-off: Smaller circuits are more interpretable but may sacrifice fidelity
• Negative masking: Zeroing out circuit components and verifying the behavior disappears
• Positive masking: Zeroing out everything except the circuit and verifying the behavior persists
• Knockout analysis: Systematically removing each circuit component to measure its marginal contribution High-fidelity circuits validate that the discovered subgraph is both necessary and sufficient for the behavior.

ACD methods rely heavily on activation patching to establish causal relationships between components. This technique intervenes in a model's forward pass by replacing a specific activation with one from a corrupted or alternative input. Variants include:

• Resampling ablation: Replacing an activation with its value from a random other input to measure the component's unique contribution
• Mean ablation: Substituting an activation with its dataset-wide average, removing all input-specific information
• Zero ablation: Setting an activation to zero, which can introduce out-of-distribution artifacts
• Path patching: Isolating the causal effect flowing along a specific path between two components while freezing all other paths These interventions provide the ground-truth signal that ACD algorithms use to score and select edges.

Subnetwork probing is an alternative ACD approach that trains binary masks over model weights to identify task-relevant subnetworks. The process involves:

• Mask parameterization: Learning a continuous mask over attention heads and MLP neurons, regularized for sparsity
• Task-specific training: Optimizing the mask so that the masked subnetwork performs well on the target behavior while the full model is frozen
• Discretization: Thresholding the learned soft masks into hard binary inclusion/exclusion decisions
• Iterative refinement: Repeating the process with progressively stronger sparsity penalties This method is particularly effective for discovering circuits that span multiple layers and involve complex interactions between attention and MLP components.

ACD fundamentally differs from traditional manual mechanistic interpretability in several key ways:

• Scalability: ACD can analyze behaviors across thousands of inputs simultaneously, while manual analysis typically examines a handful of examples
• Completeness: Algorithms systematically evaluate all possible edges, reducing the risk of missing critical components that a human might overlook
• Reproducibility: Automated methods produce deterministic results given the same hyperparameters, unlike subjective human hypothesis generation
• Limitations: ACD may find circuits that are faithful but not interpretable—the discovered subgraph reproduces the behavior but remains opaque to human understanding
• Complementary use: ACD is often used to generate candidate circuits that researchers then manually interpret and validate

The fundamental object that automated discovery algorithms seek to identify. A circuit is a sparse subgraph of connected attention heads and MLP neurons that implements a specific, human-understandable algorithm. Unlike viewing a single neuron, a circuit captures the interaction between components.

• Example: The Induction Head circuit enables in-context copying.
• Discovery methods search for the minimal set of edges and nodes that preserve a behavior.

A core causal intervention technique used to validate discovered circuits. It replaces a model's internal activation at a specific layer and token position with a value from a corrupted forward pass (e.g., with noise or a different input).

• If patching a component breaks the behavior, it is causally implicated.
• Path Patching extends this to isolate the direct effect of a specific edge between two components.

A systematic evaluation methodology for testing a hypothesized circuit. It replaces all activations outside the proposed circuit with corrupted values while keeping the circuit's internal activations intact.

• If the model's performance is fully restored, the circuit is a complete explanation.
• If performance degrades, the hypothesized circuit is missing key components, guiding further automated search.

An unsupervised architecture critical for decomposing polysemantic neurons into monosemantic features. Trained on a model's dense activations, it learns a sparse overcomplete basis.

• This transforms the discovery problem from finding circuits of neurons to finding circuits of interpretable features.
• Dictionary Learning is the broader framework for this decomposition.

The hypothesized phenomenon that neural networks represent more independent features than they have dimensions, compressing sparse features into a lower-dimensional space. This is the primary obstacle that automated circuit discovery must overcome.

• Explains why individual neurons are often polysemantic.
• Sparse autoencoders are designed to disentangle features from this compressed state.

A linear decomposition technique that attributes the final model output directly to the individual contributions of each attention head and MLP layer. It uses the residual stream's linear structure to compute additive contributions.

• Provides a first-order approximation of component importance.
• Often used as a starting point or heuristic to guide more expensive causal discovery algorithms.