Sensitivity

Sensitivity is formally defined as the directional derivative of the logit for a target class with respect to the activations at a chosen layer, taken along the unit direction of a Concept Activation Vector (CAV).

• Mathematically: S_{C,k,l}(x) = ∇h_{l,k}(f_l(x)) · v_C^l
• h_{l,k} is the logit for class k from layer l
• v_C^l is the CAV for concept C at layer l
• This measures the instantaneous rate of change in the prediction as activations are nudged toward the concept

Raw sensitivity scores alone are insufficient; TCAV applies a two-sided t-test to determine if a concept's sensitivity is statistically distinct from random noise.

• A set of random vectors is generated with the same dimensionality as the CAV
• Sensitivity scores for the concept are compared against the distribution from random vectors
• A concept is considered meaningful only if the null hypothesis is rejected (typically p < 0.05)
• This prevents spurious directions in the activation space from being misinterpreted as genuine concepts

Sensitivity is not uniform across a network's depth; it varies significantly depending on the layer at which the CAV is computed.

• Early layers: Sensitivity often reflects low-level features (textures, edges) and may show weak conceptual alignment
• Middle layers: Typically exhibit the strongest sensitivity to mid-level semantic concepts
• Final layers: Sensitivity aligns closely with class-relevant abstractions
• Analyzing sensitivity across layers reveals the hierarchical emergence of conceptual understanding in the network

A Concept Sensitivity Map visualizes which spatial regions or input features most influence the directional derivative toward a concept.

• Generated by computing the gradient of the sensitivity score with respect to the input
• Highlights pixels or tokens that, if perturbed, would most change the model's alignment with the concept
• Used to validate that sensitivity is driven by semantically relevant input regions, not spurious correlations
• Provides a visual sanity check for concept-based explanations

Sensitivity measures local responsiveness to a concept direction, while Concept Importance aggregates this into a global metric.

• Sensitivity: A per-input score; how much does this prediction change along the concept direction?
• TCAV Score: The fraction of inputs in a class whose sensitivity scores are positive for the concept
• ConceptSHAP: Uses Shapley values to assign a game-theoretic importance to each concept
• Sensitivity is the foundational measurement; importance metrics are derived from distributions of sensitivity scores across datasets

Sensitivity analysis is correlational by nature; Concept Intervention provides a causal counterpart by directly editing activations.

• Activations are modified by adding or subtracting a scaled CAV: f_l'(x) = f_l(x) + ε · v_C^l
• The resulting change in the output logit confirms whether the concept causally influences the prediction
• A concept with high sensitivity but zero causal effect may be an epiphenomenon—correlated but not causative
• Combining sensitivity with intervention yields robust concept validation