Concept Erasure

The core mechanism operates on the assumption that a concept is encoded as a linear direction in activation space. Erasure proceeds by identifying this direction—often via a linear probe trained to classify the concept—and then projecting activations onto the nullspace of that direction. This mathematically guarantees that no linear classifier can recover the erased concept from the transformed representations. The operation is typically applied to the residual stream of a transformer at a specific layer, permanently altering the information available to all downstream computations.

A critical distinction exists between linear guarding and true semantic removal. Erasure techniques guarantee that a linear classifier cannot recover the concept, but a non-linear adversary—such as a multi-layer perceptron or a fine-tuned model—may still extract it from higher-order interactions. This creates a fundamental tension:

• Stronger erasure (e.g., adversarial training against non-linear probes) degrades model utility more severely
• Weaker erasure (linear-only) preserves utility but leaves the concept vulnerable to non-linear recovery

Practitioners must calibrate the erasure strength to their specific threat model.

The effectiveness of concept erasure depends critically on where in the model the intervention is applied. Key considerations include:

• Early layers: Erasing here prevents the concept from influencing any downstream computation, but may remove information needed for unrelated tasks
• Middle layers: Often the optimal balance, as concepts are fully formed but not yet diffused into task-specific circuits
• Late layers: Minimizes collateral damage but may fail to prevent the concept from influencing intermediate reasoning steps

Causal tracing experiments are often used to identify the layer where the concept causally influences the output before applying erasure.

Concept erasure is inherently lossy. The target concept direction is rarely perfectly orthogonal to all other useful features. Removing it inevitably damages correlated information. This manifests as:

• Degradation on downstream tasks that rely on the erased concept, even indirectly
• Loss of linguistic coherence if the concept is entangled with syntactic or semantic structures
• The fairness-accuracy trade-off: Removing a protected attribute like gender often reduces the model's ability to perform coreference resolution or translation correctly

Quantifying this trade-off is essential for responsible deployment.

An extension of linear erasure that trains the projection to defeat a non-linear adversary. The method frames erasure as a minimax game:

• A predictor network attempts to recover the concept from the projected representations
• The projection matrix is optimized to minimize the predictor's accuracy

This produces representations that are robust to non-linear probing. However, the computational cost is significantly higher, and the adversarial training can be unstable. The resulting representations often exhibit worse utility preservation than linear-only methods, as the stronger constraint forces more aggressive information removal.

A diagnostic technique that trains a simple linear classifier on top of a frozen model's internal representations to test what information is encoded at a specific layer. If a linear probe can accurately predict a concept like part-of-speech or sentiment from hidden states, that concept is considered linearly separable in the representation space. This is often the first step before performing concept erasure, as it confirms the existence of a linear direction to remove.

A direction in a model's activation space that, when added to the residual stream during inference, reliably modifies high-level behavior. Unlike concept erasure which removes information, steering vectors actively induce a desired property such as truthfulness, refusal, or a specific stylistic tone. The vector is typically computed by contrasting activations from positive and negative prompt datasets.

A top-down control paradigm that reads and writes high-level cognitive vectors in a model's activation space to monitor and steer behavior without retraining. Concept erasure is a specific write operation within this broader framework. The approach treats the residual stream as a transparent medium where human-interpretable concepts can be directly manipulated, enabling real-time behavioral control.

A statistical framework for quantifying how much a model's output depends on a specific intermediate representation by measuring the effect of intervening on that representation. After erasing a concept, causal mediation analysis can verify that the targeted information no longer influences downstream predictions. This provides rigorous evidence that the erasure was effective rather than merely superficial.

The theory that neural networks represent more independent features than they have dimensions by encoding them in overlapping, nearly orthogonal directions within a shared activation space. This complicates concept erasure because a single direction may encode multiple entangled concepts. Removing a bias direction might inadvertently damage unrelated but superimposed features, requiring careful validation.

A causal intervention method that replaces a model's internal activation at a specific location with a cached activation from a different input. This technique helps localize where a concept is computed before attempting erasure. By patching activations from a corrupted input with clean activations, researchers can identify the precise layers and components responsible for encoding a target concept.