Concept Bank

The bank stores the actual CAV vectors derived from probe datasets for common model architectures and layers. This eliminates the need for every researcher to retrain linear classifiers on the same concepts.

• Architecture-specific: Separate CAVs stored for ResNet-50, InceptionV3, ViT, and other common backbones
• Layer-indexed: Vectors catalogued by layer depth, from early texture detectors to late semantic layers
• Normalized: All vectors are unit-length and stored in a consistent format for immediate use in TCAV directional derivative calculations
• Provenance metadata: Each vector records the training accuracy, dataset version, and statistical significance metrics

Concepts are organized into a hierarchical taxonomy that mirrors human semantic understanding, enabling structured navigation and relational queries across the bank.

• Hierarchical nesting: 'Dog' is a child of 'Animal' and 'Mammal', with inherited probe constraints
• Cross-referencing: Related concepts like 'striped' and 'textured' are linked for comparative analysis
• Domain-specific collections: Separate sub-banks for medical imaging concepts, autonomous driving hazards, and NLP sentiment dimensions
• Semantic similarity search: Find concepts by embedding proximity in a language model space

Every concept in the bank includes pre-computed statistical significance test results against random baselines, so users can immediately filter for concepts that are genuinely encoded in the network.

• Two-sided t-test p-values: Stored for each concept-class pair from the TCAV significance procedure
• Concept purity scores: Measures of how tightly clustered the concept's activations are relative to other concepts
• Separability indices: Quantifies how linearly separable the concept is from confounders in the activation space
• Cross-model consistency: Tracks whether a concept is reliably detected across different architectures, indicating a robust semantic encoding

The Concept Bank exposes a REST API and Python SDK for seamless integration into model auditing pipelines, enabling automated regression testing for concept sensitivity across model versions.

• Query by concept name: Retrieve CAVs, probe datasets, and metadata with a single function call
• Batch TCAV testing: Submit a model checkpoint and receive a complete concept sensitivity report
• CI/CD integration: Automatically flag when a new model version loses sensitivity to a critical concept like 'safety' or 'fairness'
• Custom concept upload: Researchers can contribute new validated concepts back to the bank with peer review

Beyond passive auditing, the bank provides pre-computed orthogonal projection matrices for concept erasure and activation steering, enabling causal intervention experiments.

• Concept erasure matrices: Projection operators that remove a concept's information from activations for fairness interventions
• Activation steering vectors: Pre-computed directions to amplify or suppress a concept during inference
• Counterfactual generation: Use concept vectors to generate 'what-if' examples where a concept is added or removed
• Causal effect benchmarks: Stored measurements of how much each concept's manipulation changes the output distribution

The fundamental building block of a concept bank. A CAV is a vector in a neural network's activation space that represents a high-level, human-understandable concept. It is derived by training a linear classifier to distinguish between examples of the concept and random counterexamples. The resulting normal vector to the decision boundary defines the concept's direction.

• Origin: Derived from the activations of any hidden layer.
• Property: Encodes a semantic abstraction, not just a visual feature.
• Usage: Forms the directional basis for sensitivity testing in TCAV.

The primary auditing technique that utilizes a concept bank. TCAV quantifies a model's sensitivity to a user-defined concept by measuring the directional derivative of a class prediction towards the CAV. It produces a score indicating how much a concept influenced a prediction.

• Process: Computes the gradient of the output with respect to activations, projected onto the CAV.
• Validation: Uses a two-sided t-test against random vectors to ensure statistical significance.
• Output: A TCAV score, a fraction of inputs positively sensitive to the concept.

An automated discovery algorithm that populates a concept bank without manual curation. ACE clusters image patches that activate similar spatial patterns in a network, treating each cluster as a candidate concept. It then uses TCAV to test the significance of these discovered concepts.

• Input: A dataset of images for a target class.
• Method: Multi-scale segmentation and activation-vector clustering.
• Result: A ranked list of automatically discovered, significant concepts for a class.

An inherently interpretable architecture that relies on a concept bank as a prediction bottleneck. A CBM first predicts a set of predefined human-understandable concepts from the input and then uses only those concept scores to make the final prediction. This forces the reasoning to be fully transparent.

• Structure: Input → Concept Predictor → Concept Scores → Final Predictor.
• Intervention: Allows direct editing of concept scores to fix errors.
• Trade-off: Interpretability is guaranteed, but accuracy may be slightly lower than an unconstrained model.

A method that applies Shapley values from cooperative game theory to quantify the importance of individual concepts from a bank for a specific prediction. ConceptSHAP fairly distributes the credit for a model's output among the set of active concepts, providing a rigorous, game-theoretic attribution.

• Mechanism: Calculates the marginal contribution of each concept across all possible concept coalitions.
• Property: Satisfies efficiency, symmetry, and null player axioms.
• Use Case: Auditing which concepts were most decisive for a single decision.

A technique for removing a specific, often sensitive, concept's information from a model's latent representation. Using a concept vector from the bank, concept erasure projects activations onto a subspace orthogonal to the concept vector, effectively deleting that semantic information.

• Operation: A linear projection matrix P = I - vv^T is applied to activations.
• Goal: Mitigate bias or forget private information without retraining.
• Verification: TCAV can be used post-erasure to confirm the concept is no longer detectable.