Concept Sensitivity Map

A Concept Sensitivity Map computes the directional derivative of a class prediction score with respect to perturbations along a Concept Activation Vector (CAV) at every spatial location in the input. This produces a heatmap where high-intensity regions indicate areas where the concept's presence strongly influences the model's output. Unlike standard saliency maps that show importance to a class, these maps show importance to a concept, enabling fine-grained semantic auditing.

Beyond input-space maps, sensitivity can be mapped across a network's computational graph. By measuring concept sensitivity at each layer's activation space, engineers can identify which layers are most responsible for encoding and transforming a concept. This profiling reveals the depth of abstraction at which a concept becomes functionally relevant, distinguishing between early-layer texture sensitivity and late-layer semantic sensitivity.

Raw sensitivity values can be noisy. Robust Concept Sensitivity Maps apply statistical significance testing—typically a two-sided t-test—to compare concept sensitivity against a null distribution of random vector sensitivities. Only regions or nodes with statistically significant sensitivity (e.g., p < 0.05 after correction) are displayed, ensuring the map reflects genuine concept encoding rather than random activation patterns.

A single input can be simultaneously analyzed for sensitivity to multiple concepts, producing comparative sensitivity maps. For example, a medical image can be mapped for sensitivity to 'edema,' 'lesion,' and 'artifact' concepts simultaneously. This allows clinicians and engineers to see which concepts compete or cooperate in a prediction, revealing the model's conceptual decision boundaries and potential reliance on spurious correlations.

An advanced form of sensitivity mapping involves causal concept intervention: directly editing activations to amplify or suppress a concept and measuring the output change. The resulting map shows not just correlation but causal influence of a concept on the prediction. This technique is critical for verifying that a model's reliance on a concept is functionally necessary, not merely correlational, for safety-critical applications.

For sequence models processing video or time-series data, Concept Sensitivity Maps extend into the temporal dimension. They highlight which frames or time steps are most sensitive to a concept, revealing when a model 'recognizes' a concept in a sequence. This is essential for debugging action recognition models or financial forecasting systems where concept timing is critical to the decision.

A matrix visualization that cross-references concepts against output classes to create a global interpretability dashboard. Each cell displays the aggregated sensitivity score, often derived from TCAV or ConceptSHAP.

• Rows represent user-defined concepts (e.g., 'fur', 'wheel', 'wing')
• Columns represent model output classes (e.g., 'cat', 'car', 'plane')
• Cell intensity and color encode the magnitude and direction of influence

This matrix allows auditors to rapidly verify that a model relies on semantically appropriate concepts for each class and flags concept leakage, where an irrelevant concept spuriously drives a prediction.

This interactive application allows an auditor to virtually intervene on a concept's activation and observe the resulting shift in the sensitivity map and final prediction. By dragging a slider to amplify or suppress a concept, the map updates in real-time.

• Tests causal influence: Does suppressing 'red' change the 'stop sign' sensitivity map?
• Validates concept disentanglement: Does modifying one concept leave others unchanged?
• Generates counterfactual explanations: 'If the concept of 'rust' were reduced, the model would classify this as a functional bridge.'

This turns a static map into a dynamic diagnostic tool for probing model reasoning.

Raw sensitivity values can be noisy. This application overlays a statistical significance mask derived from a two-sided t-test comparing concept sensitivities against random vector baselines. Only regions or layers where the concept's influence is statistically significant are highlighted.

• Filters out spurious activations that are merely artifacts of network noise
• Provides confidence bounds for the sensitivity map, crucial for high-stakes auditing
• Uses the same statistical framework as TCAV to ensure methodological consistency

This transforms a qualitative heatmap into a quantitative, auditable artifact suitable for compliance documentation.

A temporal application that tracks how a concept's sensitivity map evolves as the model is retrained or fine-tuned over time. Successive maps are compared using structural similarity indices to detect concept drift.

• Alerts auditors when a model's reliance on a critical concept (e.g., 'tumor margin') degrades
• Visualizes catastrophic forgetting of concepts during incremental learning
• Correlates sensitivity shifts with changes in validation accuracy to diagnose performance regressions

This dashboard is a key component of MLOps governance, ensuring that model updates do not silently erode conceptual alignment with domain knowledge.