Concept Intervention

Unlike correlational methods, Concept Intervention establishes a cause-and-effect relationship between a concept and a prediction. By directly editing activations, it answers 'what would the model predict if this concept were more or less present?' rather than just 'is this concept present?'

• Moves beyond feature attribution to counterfactual reasoning
• Tests hypotheses about a model's internal decision logic
• Essential for validating concept-based explanations

Intervention operates by adding or subtracting a scaled Concept Activation Vector (CAV) to the activations at a target layer. The CAV represents a direction in the high-dimensional activation space that corresponds to a human-understandable concept.

• Addition: Amplifies the concept's influence, e.g., making an image 'more striped'
• Subtraction: Diminishes or removes the concept, e.g., making text 'less formal'
• The intervention strength parameter controls the magnitude of the shift

The effect of an intervention is highly dependent on the target layer within the network. Intervening at early layers alters low-level features, while intervening at later layers modifies high-level semantic abstractions.

• Early layers: Affect texture, edges, and basic patterns
• Middle layers: Modify object parts and intermediate semantics
• Late layers: Shift categorical and relational concepts
• This allows researchers to map the hierarchy of learned features

The causal effect is quantified by measuring the change in the model's output logits or prediction probabilities before and after the intervention. A significant shift confirms the concept's causal role in the decision.

• Average Causal Effect (ACE): Mean change in prediction for a target class
• Individual Causal Effect: Change for a single input, useful for debugging
• Results are often compared against a random vector baseline to ensure the effect is not an artifact of adding noise

A specialized form of intervention removes a concept's information entirely by projecting activations onto a subspace orthogonal to the CAV. This mathematically guarantees the removal of linear information about the concept.

• Used for fairness interventions to remove sensitive attributes like gender or race
• Applied in safety research to ablate harmful concepts
• More robust than simple subtraction as it eliminates all linear traces of the concept

Concept Intervention is fundamentally different from generating counterfactual examples by modifying the input. It operates directly on the model's internal representation, bypassing the need to find a realistic input that embodies the desired conceptual change.

• Input perturbation: 'Generate a photo of the same person, but smiling'
• Concept intervention: 'Shift the internal 'smile' concept vector for this photo'
• This allows for testing concepts that are difficult to render as realistic inputs