Concept Relevance Propagation (CRP)

CRP decomposes a model's prediction by propagating relevance scores backward through the network while conditioning on specific latent concepts. Unlike standard LRP, which stops at input features, CRP reveals how high-level abstractions—such as 'stripes' or 'wheels'—contribute to the final decision. This is achieved by masking or filtering relevance flows to isolate the contribution of a target concept's subspace within a layer's activation space.

A core output of CRP is the conditional heatmap, which visualizes the relevance attributed to input features only through a specified concept. For example, a heatmap conditioned on the concept 'dog snout' shows which pixels contributed to the 'dog' classification specifically because they encode snout-related information. This provides a finer-grained explanation than unconditional saliency maps by disentangling overlapping concepts.

CRP introduces a filtering mechanism during the backward pass. At a chosen layer, the relevance signal is projected onto the subspace of a concept vector (e.g., a CAV) before continuing propagation. This ensures that only relevance that flowed through that concept's encoding is attributed to the input. The technique supports both single-concept and multi-concept conditioning for composite explanations.

Beyond visualizations, CRP provides a scalar attribution score for each concept relative to a prediction. By summing the conditioned relevance at the input level, practitioners can rank which latent concepts were most influential. This bridges the gap between qualitative heatmaps and quantitative concept importance metrics, enabling rigorous auditing of a model's internal reasoning pathways.

CRP excels at disentangling composite decisions where multiple concepts jointly influence a prediction. For instance, in classifying a 'zebra,' CRP can separately attribute relevance through the 'stripes' concept and the 'horse-like shape' concept. This decomposition reveals whether the model relies on spurious correlations or robust, domain-aligned features for its output.

CRP is designed to work synergistically with Concept Activation Vectors (CAVs) and TCAV. While TCAV measures global sensitivity to a concept, CRP localizes that sensitivity to individual inputs and pixels. The concept vectors learned via CAV methods define the subspaces used for CRP's conditional relevance filtering, creating a unified pipeline for concept-based explainability from global auditing to local instance-level interpretation.