Concept Completeness Score

A fidelity metric that evaluates the explanatory power of a concept set. The score is computed by training a concept-based surrogate model—often a linear classifier or a Concept Bottleneck Model—that uses only the concept activation values as inputs to predict the original model's output. The completeness score is the accuracy or R² of this surrogate relative to the original model. A score of 1.0 indicates the concept set fully captures the model's decision boundary; a low score reveals unexplained variance driven by concepts not yet discovered.

While Concept Importance (e.g., via ConceptSHAP or TCAV sensitivity) ranks individual concepts by their contribution, the Concept Completeness Score evaluates the set holistically. A set can contain individually important concepts yet still have low completeness if critical concepts are missing. The score answers: Have we found all the concepts that matter?

• High importance + low completeness: The known concepts are relevant but insufficient.
• Low importance + high completeness: The model's behavior is simple and fully explained by weak but exhaustive concepts.

The choice of surrogate model directly impacts the completeness estimate:

• Linear Surrogate: Tests if concepts are linearly separable for the task. A low score may indicate non-linear concept interactions rather than missing concepts.
• Concept Bottleneck Model (CBM): An inherently interpretable architecture where the bottleneck layer enforces that all information flows through predefined concepts. The CBM's accuracy relative to an unconstrained model is a direct measure of concept completeness.
• Non-linear Surrogate: A small neural network on top of concept activations can capture interactions, providing a more generous completeness estimate.

A low Concept Completeness Score triggers a diagnostic workflow:

1. Residual Analysis: Examine inputs where the surrogate disagrees with the original model. These are instances driven by unknown concepts.
2. Concept Discovery: Apply automated methods like ACE (Automatic Concept Extraction) to cluster residual activation patterns and propose new candidate concepts.
3. Concept Intervention: Causally test candidate concepts by manipulating their activations and measuring the effect on the residual error.
4. Iterative Refinement: Add validated concepts to the set and recompute the score until a satisfactory completeness threshold is reached.

The score is a critical tool for high-stakes model governance:

• Regulatory Compliance: Demonstrates to auditors that a model's behavior is exhaustively explainable through human-understandable concepts, addressing right to explanation requirements.
• Safety Assurance: In medical imaging, a completeness score of 0.98 on diagnostic concepts provides confidence that the model isn't relying on spurious correlations like hospital-specific metadata.
• Bias Detection: A completeness gap that disproportionately affects a protected subgroup may indicate the model is using prohibited concepts not included in the explanation set.

The Concept Completeness Score has important boundary conditions:

• Concept Granularity: The score depends on how concepts are defined. Overly broad concepts inflate completeness; overly narrow concepts deflate it.
• Surrogate Capacity: A weak surrogate may report low completeness even when concepts are sufficient, confusing surrogate capacity with concept insufficiency.
• Concept Interdependence: The score assumes concepts are independent inputs. Strong correlations between concepts can obscure which ones are truly necessary.
• Task Specificity: Completeness is defined relative to a specific task. A concept set complete for classifying 'dog vs. wolf' may be incomplete for 'husky vs. wolf.'

A global or local score that ranks the significance of different concepts for a model's decision-making process. While Concept Completeness Score measures the sufficiency of an entire concept set, Concept Importance attributes weight to individual concepts.

• Derived from methods like ConceptSHAP or directional derivatives
• Local importance explains a single prediction; global importance explains overall model behavior
• Essential for pruning irrelevant concepts from a bottleneck

A method that applies Shapley values from cooperative game theory to quantify the importance of individual concepts for a model's prediction. It provides a game-theoretic attribution for concept-based explanations.

• Computes the marginal contribution of each concept across all possible concept subsets
• Guarantees efficiency: importance scores sum to the prediction difference from the baseline
• Directly complements the Concept Completeness Score by validating individual concept utility

A measure of how well the representations of a single concept are clustered together and separated from other concepts in the activation space. High purity indicates strong internal consistency.

• Low purity suggests a concept vector is capturing a mixture of unrelated features
• A concept set with high completeness but low purity may produce unfaithful explanations
• Evaluated using silhouette scores or intra-cluster distance metrics

The degree to which a linear or non-linear classifier can distinguish between the activation patterns of two different concepts. High separability confirms that concepts are distinctly encoded in the network.

• Measured by Area Under the ROC Curve (AUC) of a binary concept classifier
• Poor separability between concepts inflates completeness scores artificially
• A prerequisite for building a reliable Concept Bank

An inherently interpretable architecture that first predicts a set of predefined human-understandable concepts from the input and then uses only those concept scores to make the final prediction.

• The Concept Completeness Score is a critical diagnostic for CBM design
• A low score indicates the predefined concept set is insufficient for the task
• Enables direct concept intervention by editing bottleneck values

A technique for removing a specific, often sensitive, concept's information from a model's latent representation by projecting activations onto a subspace orthogonal to the concept vector.

• Used to audit whether a concept set is complete enough to render a protected attribute irrelevant
• If erasing a concept does not degrade the completeness score, it was redundant
• Implemented via Concept Subspace Projection and iterative nullspace removal