Representation Engineering

The core insight of Representation Engineering is treating the residual stream not as a black box, but as a transparent, writable medium. Instead of modifying weights through fine-tuning, practitioners directly manipulate the high-dimensional vectors that flow between layers. This enables real-time behavioral control by adding computed steering vectors that encode abstract concepts like honesty, harmlessness, or specific task orientations.

The 'read' operation uses linear probes—simple classifiers trained on frozen internal activations—to decode what a model knows at any layer. This technique reveals:

• Factual knowledge encoded in specific layers
• Truthfulness orientation of current representations
• Emergent world models the network has constructed
• Bias or toxicity before output generation Probing provides an audit trail, transforming opaque computations into verifiable cognitive states.

The 'write' operation computes a steering vector by contrasting activations from two datasets—for example, a set of honest responses minus a set of dishonest responses. Adding this difference vector to the residual stream at a specific layer during inference reliably shifts behavior. This method, known as Contrastive Activation Addition (CAA), requires no gradient updates and can be applied dynamically, enabling per-request control policies.

Representation Engineering enables continuous cognitive telemetry. By placing probes at strategic layers, systems can detect in real-time when a model is:

• Hallucinating or entering low-confidence states
• Deceiving the user or engaging in sycophancy
• Accessing dangerous knowledge domains
• Deviating from a prescribed persona or policy This transforms safety from a static training artifact into a dynamic, observable property of the running system.

Unlike RLHF or fine-tuning, Representation Engineering operates entirely at inference time. The base model weights remain frozen. This provides critical advantages:

• Zero catastrophic forgetting of core capabilities
• Instant rollback by removing the steering vector
• Composability of multiple control vectors simultaneously
• Cost efficiency by avoiding expensive retraining cycles Control is achieved through lightweight algebraic operations on activations.

To confirm that a representation causally drives behavior—not merely correlates with it—Representation Engineering employs interchange interventions. By replacing a model's internal activation at a specific location with one from a counterfactual input, researchers can definitively localize where a computation occurs. This bridges the gap between correlational probing and true mechanistic understanding of the network's algorithms.