Directional Derivative

For a class prediction score ( f_l(x) ) at layer ( l ), the directional derivative with respect to a concept vector ( v_C ) is defined as:

( S_{C,k,l}(x) = \nabla h_{l,k}(f_l(x)) \cdot v_C )

• ( h_{l,k} ) is the logit function for class ( k ) from layer ( l )
• ( \nabla h_{l,k} ) is the gradient of that logit with respect to activations
• ( v_C ) is the unit Concept Activation Vector (CAV)
• The dot product yields a scalar sensitivity score
• A positive value indicates the concept increases the class prediction; a negative value decreases it

The directional derivative is the core computational mechanism within Testing with CAVs (TCAV). It transforms abstract concept vectors into quantifiable metrics:

• For each input ( x ) in a dataset, TCAV computes the directional derivative toward a concept vector
• The fraction of inputs with a positive directional derivative becomes the TCAV score
• This score represents how sensitive the model's prediction is to the concept
• A TCAV score of 0.8 means 80% of inputs show increased class probability when activations shift toward the concept
• Enables quantitative hypothesis testing about what a model has learned

In the high-dimensional activation space, the directional derivative has a clear geometric meaning:

• The gradient ( \nabla h ) points in the direction of steepest ascent for the class score
• The directional derivative measures the projection of this gradient onto the concept vector
• If the gradient and concept vector are aligned (cosine similarity near 1), the derivative is large and positive
• If they are orthogonal (cosine similarity near 0), the concept has no local influence
• If they are opposed (cosine similarity near -1), the concept actively suppresses the prediction
• This projection reveals which concepts the model uses as evidence for or against a class

Raw directional derivatives require statistical testing to ensure the concept signal is not an artifact:

• TCAV computes directional derivatives for the true concept vector and for multiple random vectors
• A two-sided t-test compares the distribution of sensitivity scores
• The null hypothesis: the concept vector produces scores indistinguishable from random directions
• A concept passes validation only if the p-value is below a threshold (typically 0.05)
• This guards against spurious correlations where random noise directions accidentally align with gradients
• Only statistically significant concepts are considered genuinely learned by the model

Computing directional derivatives at different network layers reveals where concepts are encoded:

• Early layers: Directional derivatives often capture low-level concepts like textures, edges, or colors
• Middle layers: Begin encoding composite concepts like object parts and shapes
• Final layers: Encode high-level semantic concepts like object categories and abstract attributes
• A concept may have a strong directional derivative at one layer but be absent at another
• This layer-wise profiling maps the hierarchy of abstraction in the network
• Engineers use this to identify which layers to probe for specific concept auditing tasks

Implementing the directional derivative requires careful numerical handling:

• Compute the gradient of the target class logit with respect to activations at the chosen layer using backpropagation
• Normalize the CAV to a unit vector to ensure comparability across concepts
• The dot product ( \nabla h \cdot v_C ) is computed per input in a batch
• For stochastic settings, average gradients over multiple forward passes
• Libraries like TensorFlow and PyTorch provide automatic differentiation for efficient gradient computation
• The result is a scalar tensor representing instantaneous sensitivity at the current activation point