Concept Selectivity

A highly selective unit activates only for its target concept and remains inert for all other inputs. This is measured by comparing the unit's activation distribution on concept-positive examples versus a broad negative dataset. A perfectly selective neuron would have a precision of 1.0—it never fires for non-concept inputs. In practice, selectivity exists on a spectrum, with top-performing units achieving >0.95 precision on curated probe datasets. This property is essential for building trustworthy concept-based explanations, as it ensures that when a concept is attributed to a prediction, the signal is not contaminated by unrelated features.

Selectivity is the inverse of polysemanticity—the phenomenon where a single neuron responds to multiple unrelated concepts. In large language models, polysemantic neurons are the norm, with individual units activating for everything from curly braces in code to biological terms. Concept selectivity metrics help researchers identify the rare monosemantic units that encode a single, coherent idea. Techniques like sparse autoencoders are explicitly designed to decompose polysemantic activations into a set of highly selective, monosemantic features, improving the interpretability of the model's internal representations.

Selectivity is empirically measured by training a linear probe to distinguish a concept from random counterexamples using a unit's activations. The probe's classification accuracy on a held-out test set serves as the selectivity score. A score of 1.0 indicates perfect linear separability—the concept direction is cleanly encoded. This method is foundational to the TCAV (Testing with CAVs) framework, where the CAV itself is the weight vector of a linear classifier. The statistical significance of the selectivity score is then validated against a null distribution of random directions to ensure the concept is genuinely learned.

Concept selectivity is not uniform across a network's depth. Early layers typically exhibit low selectivity, encoding basic textures and edges. Middle layers show peak selectivity for mid-level concepts like shapes and object parts. Final layers often return to lower selectivity as representations compress into task-specific features. Mapping this selectivity profile across layers reveals where abstract semantic understanding emerges. For example, in vision models, ImageNet class selectivity peaks in the penultimate layer, while in language models, syntactic concept selectivity peaks earlier than semantic concept selectivity.

Correlational selectivity is necessary but insufficient. Causal selectivity requires demonstrating that clamping or ablating a unit's activation directly and predictably changes the model's output for the concept. This is tested through activation patching or knockout experiments. A causally selective unit for 'stripes' will, when ablated, degrade the model's ability to recognize zebras but not horses. This causal criterion distinguishes genuine concept encoding from spurious correlations and is the gold standard for mechanistic interpretability claims.

Rather than merely measuring selectivity post-hoc, it can be optimized during training. Concept bottleneck models enforce selectivity by design—each bottleneck neuron is assigned to a single predefined concept. Concept whitening modules replace batch normalization layers to align latent axes with concepts, maximizing per-axis selectivity. Adding a selectivity loss term that penalizes a neuron's response to non-target concepts encourages the emergence of disentangled, monosemantic representations. This proactive approach yields models that are interpretable by construction, not just by analysis.

A measure of how well the representations of a single concept are clustered together and separated from other concepts in the activation space. High purity indicates that a neuron or direction responds to one concept and not to others.

• Evaluated using silhouette scores or Davies-Bouldin indices
• A pure concept direction has low intra-class variance and high inter-class separation
• Directly quantifies the degree of concept selectivity in a latent space

The degree to which a linear or non-linear classifier can distinguish between the activation patterns of two different concepts. High separability confirms that concepts are encoded in distinct, non-overlapping regions.

• Measured by training a linear probe on concept-positive vs. concept-negative examples
• A high ROC-AUC score indicates strong selectivity
• Low separability suggests the network uses entangled or polysemantic representations

A technique for removing a specific concept's information from a model's latent representation by projecting activations onto a subspace orthogonal to the concept vector. This directly tests causal selectivity.

• Uses INLP (Iterative Nullspace Projection) or gradient-based methods
• If erasing a concept degrades only that concept's classification, the representation was selective
• Used to remove sensitive attributes like gender or race from representations

A module that replaces a standard batch normalization layer, aligning the latent space axes with predefined concepts to produce a disentangled and interpretable representation.

• Enforces axis-aligned selectivity where each dimension corresponds to exactly one concept
• Applies a whitening transformation followed by an orthogonal rotation
• Guarantees that concept dimensions are mutually orthogonal and independently manipulable

The act of directly modifying a model's internal activations during inference to increase or decrease the presence of a concept. This causally tests whether a direction is selectively responsible for a single concept.

• Performed by adding or subtracting a scaled Concept Activation Vector (CAV)
• If intervening on a concept direction changes only that concept's output, selectivity is confirmed
• Reveals spurious correlations when interventions produce unintended side effects

A metric that evaluates how sufficient a set of discovered or defined concepts is for explaining a model's full behavior. It measures whether the selected concepts completely capture the model's decision logic.

• Computed by comparing predictions from a Concept Bottleneck Model (CBM) to the original model
• Low completeness indicates missing concepts or reliance on non-interpretable features
• Complements selectivity by ensuring the model doesn't ignore important concepts entirely