Concept-Based Explanation

Operates on high-level concepts rather than low-level features. Instead of explaining a prediction in terms of pixel intensities or token IDs, these methods use human-interpretable abstractions like 'stripes,' 'wheels,' or 'texture.' This bridges the gap between a model's internal geometry and a domain expert's vocabulary, making explanations actionable for non-ML stakeholders.

Quantifies how much a model's prediction changes when activations are perturbed along a concept vector direction. The directional derivative measures the model's sensitivity to a specific concept, revealing whether the network relies on that abstraction for its decision. Statistical significance testing against random vectors ensures the sensitivity is not an artifact.

Relies on the empirical finding that semantically meaningful concepts are often encoded as linearly separable directions in a neural network's activation space. A simple linear classifier can distinguish between examples of a concept and random counterexamples, producing a Concept Activation Vector (CAV) that serves as the axis of explanation.

Supports both global model auditing and local prediction explanation:

• Global: Identify which concepts a model consistently relies on across an entire class
• Local: Determine which specific concepts drove a single prediction This dual granularity serves both compliance officers reviewing systemic behavior and engineers debugging individual edge cases.

Enables concept intervention—directly modifying internal activations during inference to increase or decrease a concept's presence. By observing the resulting change in output, practitioners can establish causal relationships between concepts and predictions, moving beyond correlation to verify that the model genuinely uses the concept in its reasoning chain.

Methods like Automatic Concept Extraction (ACE) discover meaningful concepts without manual specification. ACE clusters input patches that produce similar activation patterns, then validates discovered concepts using TCAV. This automation scales concept-based auditing to large models where manual concept definition would be prohibitively labor-intensive.

A vector in a neural network's activation space that represents a high-level, human-understandable concept. It is derived by training a linear classifier to distinguish between examples of the concept and random counterexamples. The resulting vector direction encodes the concept's semantic meaning within the model's internal representation.

A technique that quantifies a model's sensitivity to a user-defined concept by measuring the directional derivative of a class prediction towards the Concept Activation Vector. It uses a two-sided t-test to determine if sensitivity scores are statistically significant compared to random vectors, ensuring the concept is not an artifact.

An inherently interpretable architecture that first predicts a set of predefined human-understandable concepts from the input and then uses only those concept scores to make the final prediction. This forces the model to reason through explicit abstractions, enabling direct intervention on intermediate concepts.

A method that applies Shapley values to quantify the importance of individual concepts for a model's prediction. It provides a game-theoretic attribution for concept-based explanations, fairly distributing credit among all concepts present in the input to explain the final output.

An extension of Layer-wise Relevance Propagation that decomposes a model's decision not just by input features but also through higher-level latent concepts. It traces the flow of relevance through the network's layers, attributing importance to specific concepts at each level of abstraction.

A technique for removing a specific, often sensitive, concept's information from a model's latent representation. It works by projecting activations onto a subspace orthogonal to the concept vector, effectively zeroing out the model's ability to detect or use that concept while preserving other information.