Causal Abstraction

The primary experimental method for testing causal abstraction. It works by replacing the activation of a target model component processing a base input with the activation from a source input. If the model's output changes exactly as predicted by the high-level causal graph, the component is considered causally aligned with that variable. This technique directly validates whether a neural network implements a hypothesized symbolic algorithm.

A statistical framework for quantifying how much a model's output depends on a specific intermediate representation. It measures the average causal effect of intervening on a neuron or attention head. Key metrics include:

• Indirect Effect: The portion of the output change mediated through the target component.
• Direct Effect: The residual influence bypassing the component. This analysis is foundational for building the causal graphs that abstraction aims to align with.

A localized causal intervention method that replaces a model's internal activation at a specific location with a cached activation from a different forward pass. By patching activations from a corrupted input into a clean run (or vice versa), researchers can precisely localize where a specific computation occurs. This technique is often used to identify the minimal set of components that must be aligned for a successful causal abstraction.

The core data pathway in a transformer architecture where each layer reads from and writes additive updates to a running hidden state. Causal abstraction treats the residual stream as the primary medium where high-level variables are encoded. Interventions and alignment are typically performed on the residual stream states at specific layers, as this is where the model composes its intermediate representations.

The broader research field that encompasses causal abstraction. Its goal is to reverse-engineer the internal computations and learned algorithms within a neural network's weights. Causal abstraction provides the rigorous, causal proof that a hypothesized mechanism is actually implemented, moving beyond correlational probing to establish a definitive mapping between model components and symbolic functions.

A destructive causal technique that removes or zeroes out a model component, such as a neuron or attention head, to measure the resulting drop in performance. In the context of causal abstraction, ablation is used to validate the necessity of a component for a specific computation. If ablating a component breaks the alignment with a causal variable, it confirms the component's functional role in the hypothesized algorithm.