Representation Engineering

Representation Engineering operates directly on a model's hidden states—the high-dimensional vector representations of concepts formed during forward passes. By identifying the specific directions in activation space that encode harmful concepts like deception or toxicity, safety vectors can be added or subtracted to steer generation. This is fundamentally different from prompt-based defenses, as it controls the model's internal cognitive state rather than its external inputs. The technique leverages the linear representation hypothesis, which posits that high-level concepts are encoded as linear directions in a model's representation space.

Safety vectors are computed by contrasting model activations on paired datasets of harmful and harmless prompts. The process involves:

• Collecting hidden states from a target layer when processing harmful prompts
• Collecting hidden states from the same layer when processing harmless prompts
• Computing the principal component or mean difference between these two distributions
• The resulting vector represents the 'harmful concept direction' in activation space This vector can then be applied during inference to push activations away from harmful regions without any gradient updates to the model weights.

Unlike fine-tuning or RLHF, which permanently alter model weights, Representation Engineering applies dynamic control vectors during the forward pass. This enables:

• Zero-shot application: A single computed vector can generalize to suppress unseen harmful behaviors
• Granular modulation: The strength of the intervention can be scaled by adjusting the vector's coefficient
• Selective application: Different vectors can be applied for different safety domains (toxicity, deception, bias) simultaneously
• No catastrophic forgetting: Since weights remain unchanged, general capabilities are preserved without the safety alignment tax

Representation Engineering requires identifying which transformer layers encode the target concept most effectively. Research shows that:

• Middle layers typically encode high-level semantic concepts most linearly
• Early layers process token-level features and syntax
• Later layers shift toward next-token prediction probabilities Intervening at the optimal layer maximizes the trade-off between safety enforcement and output quality. Tools like linear probing classifiers are used to measure concept separability at each layer, guiding the selection of the intervention point.

Representation Engineering is the broader methodological framework, while Activation Steering is the specific inference-time technique that applies the computed vectors. The field encompasses:

• Reading: Extracting and interpreting what concepts are represented in hidden states
• Writing: Adding control vectors to modify those representations
• Monitoring: Detecting when harmful concepts activate, even if the output appears safe This tripartite approach enables not just suppression of harmful outputs, but also auditing of the model's internal state for latent harmful intent that might emerge through jailbreak attempts.

A key finding in Representation Engineering is that safety vectors exhibit cross-model transferability. Vectors computed on one model can often be applied to different models within the same architecture family, and sometimes across different architectures entirely. This suggests that foundation models converge to similar internal representations of abstract concepts. The practical implication is that safety vectors can be developed once and deployed across model updates or fine-tuned variants, dramatically reducing the cost of maintaining safety alignment across a model fleet.