Sparse Autoencoder

The core objective of a sparse autoencoder is to apply an L1 penalty to the hidden layer activations during training. This forces the model to reconstruct the input activation vector using only a small number of active latent dimensions. By constraining the representation, the autoencoder acts as a decompressor, separating superposed features that were previously compressed into a single dense vector. The resulting sparse code reveals the independent, underlying concepts the network is processing at that specific layer.

A sparse autoencoder consists of two learned linear transformations:

• Encoder (W_enc): Maps the model's dense activation vector (d_model dimensions) to a higher-dimensional, sparse hidden representation (d_hidden >> d_model) using a ReLU activation to enforce non-negativity.
• Decoder (W_dec): Maps the sparse hidden representation back to the original activation space to produce a reconstruction. The columns of the decoder matrix are the dictionary features, each representing a distinct, interpretable concept. The reconstruction error quantifies how much information was lost in the sparse decomposition.

A major motivation for sparse autoencoders is to resolve polysemanticity, where a single neuron responds to multiple unrelated concepts. By expanding the dimensionality, the autoencoder can assign a unique, monosemantic feature direction to each concept. For example, a single neuron that fires for both 'academic citations' and 'legal summons' can be decomposed into two separate dictionary features, each activating exclusively for one meaning. This provides a clean, interpretable basis for analyzing the model's internal knowledge.

Once a sparse autoencoder is trained, each dictionary feature can be interpreted by finding the input examples that cause it to activate most strongly. This is done by:

• Maximally Activating Examples: Scanning a large dataset to find inputs that produce the highest activation value for a specific feature.
• Logit Weight Analysis: Examining the direct effect of a feature on the output logits via the decoder matrix, revealing what tokens the feature promotes. This process allows researchers to build a catalog of human-understandable concepts represented within the model.

Sparse autoencoder features are not just descriptive; they can be used for causal intervention. By artificially clamping a specific feature's activation to a high value during a forward pass, researchers can steer the model's behavior in a predictable way. For instance, activating a feature for 'optimism' can alter the sentiment of generated text. This technique, a form of activation engineering, provides strong evidence that the discovered features are causally responsible for the model's outputs, not merely correlated with them.

Sparse autoencoders are typically trained on the residual stream activations at specific layers of a transformer model. The residual stream is the central information highway where all previous layers write their output. Training autoencoders at different layers reveals the hierarchical nature of features, from low-level syntactic patterns in early layers to high-level semantic concepts in later layers. A key challenge is scaling these methods to frontier models, requiring extremely large and efficient autoencoder architectures to capture the full breadth of features.

The mathematical framework underlying sparse autoencoders. A method for decomposing a model's activations into a sparse linear combination of learned basis vectors, each representing a distinct, interpretable feature.

• Learns an overcomplete basis where the number of features exceeds the activation dimension
• Enforces sparsity via an L1 penalty on the encoded representation
• Each basis vector ideally corresponds to a single monosemantic concept
• Directly addresses the superposition hypothesis by disentangling compressed features

A hypothesized phenomenon where a neural network represents more independent features than it has dimensions in a given layer. The model compresses sparse features into a lower-dimensional space by exploiting the fact that features are rarely active simultaneously.

• Explains why individual neurons appear polysemantic
• Sparse autoencoders are designed to reverse this compression
• Mathematically formalized by the toy models of superposition research agenda
• Critical motivation for using overcomplete dictionaries in mechanistic analysis

The property of a single neuron or feature direction corresponding to exactly one natural, human-interpretable concept. A monosemantic neuron fires exclusively for a specific idea, such as 'Arabic text' or 'curly quotation marks.'

• The ideal outcome of sparse autoencoder training
• Contrasts sharply with polysemanticity, where one neuron responds to multiple unrelated concepts
• Verified by examining max activating examples for each learned feature
• Enables reliable causal intervention on specific concepts

An optimization-based technique that synthesizes input examples that maximally activate a specific neuron, channel, or learned dictionary feature. For sparse autoencoder features, this reveals what concept the feature encodes.

• Uses gradient ascent on the input to maximize feature activation
• Incorporates diversity regularization to show varied aspects of the feature
• Produces human-interpretable visualizations for vision models
• For language models, shows top activating tokens and contexts instead

A systematic evaluation methodology that tests a hypothesized circuit or feature decomposition by replacing all activations outside the circuit with corrupted values and verifying the model's performance is preserved.

• If the sparse autoencoder features truly capture the model's computation, scrubbing non-feature activations should not degrade performance
• Provides rigorous falsification criteria for interpretability claims
• Distinguishes between correlational and causal feature importance
• Essential for validating that dictionary features are functionally meaningful

A variant of a sparse autoencoder trained on the activations of two different models simultaneously to identify shared and distinct features between them. Enables comparative mechanistic analysis across model checkpoints or architectures.

• Learns a shared dictionary of features across both models
• Identifies features that are universal versus model-specific
• Tracks how features evolve during training by comparing checkpoints
• Useful for studying feature emergence and grokking phenomena