Statistical Significance Testing

The core mechanism for validating a Concept Activation Vector (CAV). TCAV computes sensitivity scores for a concept across multiple images, then performs a two-sided t-test comparing these scores against a null distribution generated from thousands of random direction vectors. A concept is considered statistically significant if the resulting p-value falls below a threshold (typically 0.05), rejecting the null hypothesis that the concept's sensitivity is indistinguishable from random noise.

The validity of the test hinges on a robust null distribution. This is constructed by:

• Generating hundreds or thousands of random vectors in the same activation space as the CAV.
• Computing the directional derivative for each random vector across the same set of test inputs.
• Building a null distribution of sensitivity scores that represents the expected influence of a meaningless direction. This process controls for the inherent variance in model gradients and prevents false positives from spurious correlations.

When testing dozens or hundreds of concepts simultaneously, the probability of a Type I error (false positive) increases. The Bonferroni correction adjusts the significance threshold by dividing the desired alpha level (e.g., 0.05) by the number of independent tests performed. For example, testing 20 concepts would require a p-value below 0.0025 for significance. This conservative adjustment ensures that only concepts with robust, reproducible influence are reported as significant.

The raw metric fed into the t-test is the sensitivity score, defined as the fraction of inputs for which the directional derivative towards the CAV is positive. A concept that consistently increases the target class logit across many examples will have a high sensitivity score. The t-test evaluates whether the mean sensitivity of the true concept is significantly greater than the mean sensitivity of random vectors, accounting for the variance in both distributions.

In the ACE algorithm, statistical significance testing serves as a critical filter. After clustering image patches to discover candidate concepts, ACE runs TCAV with a two-sided t-test on each cluster. Only clusters that achieve statistical significance are retained as valid, human-interpretable concepts. This automated gating mechanism prevents the concept bank from being polluted by incoherent visual patterns that the model does not actually use for prediction.

A concept can be statistically significant yet practically irrelevant. The effect size—the magnitude of the difference between the concept's mean sensitivity and the random baseline—must also be considered. A concept with a tiny effect size might pass a t-test with a large sample size but have negligible influence on the model's decisions. Robust TCAV analysis reports both the p-value and the Cohen's d or raw sensitivity difference to distinguish between statistical and practical significance.

The specific statistical test used in TCAV to compare concept sensitivity scores against a null distribution of random sensitivity scores. A two-sided test evaluates whether the concept's mean sensitivity is significantly different from random—either higher or lower—without assuming directionality. The test produces a p-value indicating the probability that the observed difference occurred by chance. If p < 0.05, the concept is deemed statistically significant. This approach accounts for the natural variance in sensitivity measurements across different random vector samples.

The formal assumption that a concept activation vector (CAV) has no meaningful relationship with the target class prediction—meaning the concept's sensitivity scores are indistinguishable from those of random direction vectors. TCAV constructs the null distribution by generating hundreds of random vectors in the activation space and computing their sensitivities. The null hypothesis is rejected when the two-sided t-test yields a statistically significant result, providing evidence that the concept is genuinely encoded in the model's representations rather than being a spurious correlation.

A critical adjustment applied when testing many concepts simultaneously against a single model. Without correction, the probability of false positives inflates with each additional test—testing 20 concepts at p < 0.05 yields roughly one false positive by chance alone. TCAV implementations typically apply Bonferroni correction (dividing the significance threshold by the number of tests) or Benjamini-Hochberg procedures to control the false discovery rate. This ensures that reported significant concepts maintain statistical validity across the entire concept bank.

The foundation of TCAV's statistical framework: generating a distribution of sensitivity scores from isotropic random vectors uniformly sampled from the activation space. These random directions serve as the control group, representing what sensitivity looks like when there is no meaningful concept alignment. Key properties include:

• Dimensionality matching: Random vectors have the same dimension as the CAV
• Sample size: Typically 500-1000 random vectors for stable null distributions
• Independence: Each vector is drawn independently to avoid distribution bias

Beyond binary significance testing, TCAV can report Cohen's d or similar effect size metrics to quantify the magnitude of a concept's influence. While a p-value indicates whether an effect exists, effect size measures how substantial it is. A concept might be statistically significant with a tiny effect size if tested on enough examples, but practically meaningless. Reporting both metrics provides a more complete picture: statistical significance confirms the concept is real, while effect size reveals its practical importance to the model's decision-making.