Inference-Time Intervention

A lightweight control method that adds a pre-computed bias vector to the residual stream at a specific layer to induce a desired behavior.

• The steering vector is computed offline from a contrastive dataset (e.g., positive vs. negative sentiment prompts)
• Addition occurs at a single targeted layer, typically identified through causal tracing
• Induces behaviors such as refusal suppression, truthfulness enhancement, or sentiment control
• Does not require retraining or fine-tuning; purely an inference-time operation
• Related to Contrastive Activation Addition (CAA), which computes the vector as the difference between mean activations of two prompt sets

A causal intervention method that replaces a model's internal activation at a specific location with a cached activation from a different input.

• Used to localize where a computation occurs within the network
• Three variants: clean patching (restoring clean activations into a corrupted run), corrupted patching (injecting noise into a clean run), and cross-input patching (swapping activations between two different inputs)
• Critical for circuit discovery — identifying the minimal subgraph of components responsible for a behavior
• Operates on residual stream states, attention head outputs, or MLP layer outputs
• Forms the experimental backbone of mechanistic interpretability research

An inference-time search heuristic that maximizes the probability difference between an expert model and an amateur model to generate more factual and coherent text.

• The amateur model is typically a smaller or earlier-checkpoint version of the expert
• Logits are combined as: logit_expert − α × logit_amateur before softmax
• Suppresses common but uninformative tokens (e.g., generic continuations) that both models assign high probability
• DoLa Decoding is a self-contained variant that contrasts later transformer layers against earlier layers within the same model
• Reduces hallucinations and improves factuality without external knowledge retrieval

A direction in a model's activation space that, when added to the residual stream during inference, reliably modifies the model's high-level behavior.

• Discovered through Representation Engineering, a top-down paradigm for reading and writing cognitive vectors
• A single vector can encode complex behaviors: truthfulness, refusal, corrigibility, or power-seeking tendency
• Applied by adding the scaled vector to the residual stream at every token position across one or more layers
• The scaling coefficient controls the intervention strength — higher values produce stronger behavioral shifts
• Enables real-time monitoring and control of model internals without weight modification

A method for identifying the specific hidden states within a transformer that are causally responsible for recalling a fact.

• Procedure: corrupt the input (e.g., add noise to the subject token), then systematically restore clean activations from the uncorrupted forward pass at each layer and token position
• Measures the indirect effect — how much each restored state increases the probability of the correct factual answer
• Revealed that factual recall in LLMs is localized to a small set of MLP layers at the last subject token position
• Foundational technique behind the ROME (Rank-One Model Editing) method for precise weight updates
• Distinguishes between states that merely encode information versus those that propagate it causally

A decoding technique that applies the unembedding matrix directly to intermediate residual stream states, allowing researchers to inspect a model's iterative token predictions before the final layer.

• Bypasses all subsequent layers to reveal what the model "believes" at each processing stage
• Early layers typically produce nonsensical tokens; intermediate layers reveal gradual refinement toward the final answer
• Used to study how predictions evolve through the network and where specific knowledge emerges
• A variant called the tuned lens learns an affine transformation per layer to improve intermediate decoding fidelity
• Provides a window into the progressive computation occurring across transformer depth