Self-Repair

A causal intervention technique used to isolate functional circuits and verify self-repair. By replacing a model's internal activation at a specific layer and token position with a cached activation from a different forward pass, researchers can observe if later layers dynamically adjust to compensate for the perturbation. This is the primary experimental method for proving that a network is not just failing gracefully but actively re-routing computation.

The primary information highway in a transformer where self-repair is most observable. Each layer reads from and writes to this shared accumulating state. When a component is ablated, the residual stream carries the 'damaged' representation forward. Self-repair occurs when subsequent layers read this corrupted state and write a corrective vector that restores the necessary information, effectively bypassing the removed component through the OV circuit of a later attention head.

The process of identifying the minimal subgraph of a network's computational graph responsible for a behavior. In the context of self-repair, circuit analysis reveals redundant pathways. It often uncovers that a specific function is not localized to a single head or neuron but is computed by a distributed circuit. When one node is ablated, a backup circuit with overlapping functionality in a later layer activates to maintain performance.

An unsupervised technique used to decompose a model's dense, polysemantic activations into a sparse set of interpretable, monosemantic features. SAEs are crucial for understanding self-repair at the feature level. They can show that after an ablation, the latent representation of a 'lost' feature may spontaneously re-emerge in a later layer, encoded by a different set of dictionary elements, proving the model has a robust, distributed representation of the concept.

A formal hypothesis-testing framework for verifying if a proposed circuit explains a model's behavior. To test a self-repair hypothesis, an analyst can systematically resample activations. If the hypothesized redundant component is truly responsible for the repair, replacing its activations with those from a corrupted run will break the repair, while replacing other activations will not. This provides rigorous evidence for the specific mechanism of dynamic compensation.

The theory that neural networks represent more independent features than they have dimensions by encoding them in almost-orthogonal directions. This is the fundamental reason self-repair is possible. Because features are stored in superposition, the removal of one neuron does not cleanly delete a single feature. Instead, it slightly corrupts many features, and the model's learned weights in later layers can compute interference-correcting transformations to recover the partially damaged information.