Singular Value Decomposition (SVD)

SVD factorizes a weight matrix W into UΣVᵀ. In the OV circuit (W_O W_V), the right singular vectors (columns of V) represent the directions in the residual stream that the head reads from, while the left singular vectors (columns of U) represent the directions it writes to. The singular values in Σ quantify the strength of each read-write pair. For QK circuits (W_Qᵀ W_K), SVD reveals the principal query-key directions that dominate attention score computation.

The top singular vectors of an attention head's OV matrix often correspond to interpretable semantic operations:

• Copying behavior: A head that copies token information will have a dominant singular vector aligned with the identity-like mapping in the residual stream.
• Inhibition: Negative singular values (or vectors with opposing signs) can reveal how a head suppresses specific token representations.
• Concept subspaces: The span of the top-k singular vectors defines the low-dimensional subspace in which the head performs its primary function, filtering out noise.

The effective rank of a weight matrix—the number of singular values above a noise threshold—indicates how many independent read-write operations an attention head performs:

• Rank-1 heads: Often correspond to simple copying or primitive operations. Induction heads frequently exhibit near rank-1 OV matrices.
• High-rank heads: Perform more complex, compositional transformations, potentially multiplexing multiple functions.
• Singular value decay: A sharp drop-off after the first few singular values suggests the head's behavior is dominated by a small number of principal directions, making it amenable to mechanistic interpretation.

SVD enables precise surgical interventions on model weights:

• Rank-one updates: The ROME (Rank-One Model Editing) technique uses SVD to insert new factual associations into MLP layers by adding a rank-one matrix to W_proj, modifying only the singular vectors associated with the target knowledge.
• Pruning: Zeroing out small singular values compresses the matrix while preserving its dominant behavior, useful for removing redundant or noisy components.
• Subspace projection: Projecting activations onto or away from specific singular vector subspaces allows causal testing of whether a particular direction is necessary for a behavior.

The canonical correlation or subspace alignment between the singular vectors of different attention heads quantifies functional similarity:

• Heads with highly aligned top singular vectors likely perform redundant or synergistic operations.
• Orthogonal singular vector subspaces indicate heads that specialize in distinct, non-interfering functions.
• This alignment metric is used in circuit discovery to cluster heads into functional groups and identify the minimal set of heads required for a specific task.

Applying SVD to the embedding matrix W_E and unembedding matrix W_U reveals the principal semantic axes of the token vocabulary:

• The singular vectors of W_E define the directions along which token semantics vary most significantly.
• The Logit Lens technique effectively projects intermediate residual stream states onto the singular vectors of W_U to interpret the model's evolving predictions.
• The alignment between W_E and W_U singular vectors (often tied via weight tying) shows how the model maps input semantics to output logits through a shared representational space.