Activation Addition

Activation addition operates purely during the forward pass, leaving the model's original weights completely untouched. This makes it a zero-shot control method that requires no gradient updates, no retraining, and no access to the training data. The intervention is applied by simply adding a pre-computed steering vector to the residual stream at a chosen layer, immediately altering downstream computations. This property makes it ideal for rapid experimentation and deployment scenarios where model integrity must be preserved.

The core mechanism relies on computing a direction in activation space that corresponds to a desired behavior. Common computation methods include:

• Contrastive Activation Addition (CAA): Subtracting mean activations from a negative-prompt dataset (e.g., harmful requests) from mean activations of a positive-prompt dataset (e.g., harmless requests)
• Difference-in-means: Computing the vector between two sets of cached activations representing opposing behaviors
• Linear probing weights: Using the learned weights of a linear classifier trained to detect a concept as the steering direction The resulting vector encodes the high-level cognitive shift to be induced.

The effectiveness of activation addition depends critically on which layer receives the intervention. Different layers in a transformer encode different levels of abstraction:

• Early layers: Encode low-level syntactic and positional features
• Middle layers: Represent semantic content and factual associations
• Later layers: Refine predictions toward final output distributions Injecting a steering vector at the optimal layer for a target behavior—such as a refusal vector at the layer where ethical reasoning crystallizes—maximizes control while minimizing unintended side effects on unrelated capabilities.

Activation addition can induce a wide range of behavioral shifts by targeting different cognitive dimensions in activation space:

• Refusal vectors: Suppress harmful or policy-violating outputs
• Sentiment vectors: Steer text generation toward positive or negative emotional tone
• Honesty vectors: Increase truthfulness and reduce hallucination
• Style vectors: Shift writing style, formality, or persona
• Language vectors: Bias generation toward a specific language Each vector represents a distinct, composable axis of control that can be applied independently or combined.

Activation addition is a core technique within the broader Representation Engineering (RepE) paradigm. While RepE encompasses both reading (monitoring) and writing (controlling) cognitive vectors in activation space, activation addition specifically implements the writing operation. It shares conceptual foundations with:

• Steering vectors: The general class of directional interventions
• Concept erasure: Removing unwanted directions from representations
• Causal mediation analysis: Identifying which representations causally influence outputs Together, these techniques form a top-down control framework that treats model internals as a programmable interface.

Advantages:

• Zero-shot: No training data or fine-tuning required
• Composable: Multiple vectors can be added simultaneously
• Reversible: Removing the addition restores original behavior
• Computationally cheap: Single vector addition per forward pass

Limitations:

• Layer sensitivity: Wrong layer choice degrades performance
• Vector magnitude tuning: Too strong an addition causes output degradation
• Concept overlap: Steering one behavior may inadvertently affect correlated behaviors
• Context dependence: A vector computed on one distribution may not generalize perfectly to others