Concept-Based Explanations

CAVs represent a direction in the model's activation space that corresponds to a human-friendly concept (e.g., 'stripes', 'doctor', 'confidential'). The technique, formalized in TCAV (Testing with CAV), measures how sensitive a prediction is to a concept by calculating the directional derivative of the model's output toward the CAV.

• How it works: A linear classifier is trained to distinguish between activations of concept examples and random counterexamples
• Key metric: TCAV score quantifies the fraction of inputs where the concept positively influenced the prediction
• Example: A dermatology model's 'irregular border' concept should strongly influence melanoma classification

CBMs are architectures that force the model to predict human-specified concepts as an intermediate layer before making the final prediction. This creates an inherent bottleneck where the model must first recognize concepts like 'bone spurs' or 'joint space narrowing' before diagnosing osteoarthritis.

• Training: Requires datasets annotated with both input features and concept labels
• Intervention: Domain experts can manually correct mispredicted concepts at test time to improve final accuracy
• Trade-off: Slight accuracy reduction in exchange for complete concept-level transparency

Unlike CBMs which bake concepts into the architecture, post-hoc methods discover concepts from a trained model's internal representations without architectural modification. Techniques include clustering activations, matrix factorization, and probing with concept datasets.

• ACE (Automatic Concept Extraction): Aggregates image patches that strongly activate specific filters, then clusters them into meaningful concepts
• Network Dissection: Aligns individual neurons with human-labeled visual concepts by measuring intersection-over-union scores
• Limitation: Discovered concepts may not perfectly align with human-interpretable categories

This approach structures explanations as logical combinations of concepts using formal operators like AND, OR, and NOT. Instead of just listing relevant concepts, the system explains that a decision was made because 'concept A AND concept B were present, but concept C was absent.'

• Form: If-then rules over concept presence/absence (e.g., 'IF stripes AND fangs THEN venomous')
• Advantage: Produces verifiable, rule-based justifications that align with expert diagnostic criteria
• Challenge: Real-world concepts often have fuzzy boundaries that resist strict logical formalization

Quantifies how much each concept contributed to a specific prediction, analogous to feature attribution but operating at the concept level. Methods extend SHAP, Integrated Gradients, or attention weights to concept dimensions rather than raw input tokens.

• ConceptSHAP: Adapts Shapley values to assign importance to concepts by masking concept activations
• Use case: A loan denial explanation showing 'debt-to-income ratio' contributed 60% while 'employment stability' contributed 25%
• Output: Ranked list of concepts with normalized contribution percentages

Extends concept-based explanations across multiple data modalities by aligning concepts in a shared semantic space. A concept like 'aggressive driving' can be grounded simultaneously in video (sudden lane changes), audio (engine revving), and telemetry (rapid acceleration).

• Cross-modal alignment: Uses contrastive learning to map concepts from different modalities into a joint embedding space
• Application: Autonomous vehicle incident explanations referencing visual, auditory, and sensor concepts simultaneously
• Requirement: Paired multimodal datasets with concept annotations across all modalities