Automatic Concept Extraction (ACE)

ACE operates by segmenting images from a target class at multiple resolutions—from fine-grained patches to larger superpixels—to capture concepts at varying levels of abstraction. This hierarchical approach ensures that both textural details (e.g., stripes on a zebra) and structural components (e.g., a wheel on a car) are represented in the discovered concept bank.

Each segmented image patch is fed through the network to extract its activation vector at a bottleneck layer. ACE then applies k-means clustering in this high-dimensional activation space, grouping patches that trigger similar spatial firing patterns. The resulting cluster centroids become candidate concept vectors, representing recurring visual motifs the network has learned to detect.

Not all clusters represent meaningful concepts—some are artifacts of noise or spurious correlations. ACE addresses this by using Testing with CAVs (TCAV) to evaluate each candidate. A linear classifier is trained to distinguish cluster patches from random counterexamples, and a two-sided t-test determines whether the concept's sensitivity scores significantly diverge from random baselines.

Validated concepts are ranked by their TCAV score—the fraction of target-class images where the concept positively influences the prediction. This produces an ordered list of the most salient visual concepts driving the model's decisions, enabling researchers to audit whether the network relies on domain-appropriate features or exploits dataset biases.

The quality of discovered concepts depends critically on the bottleneck layer chosen for activation extraction. Lower layers yield texture-level concepts (edges, colors), while higher layers capture semantic abstractions (object parts, shapes). ACE requires careful layer selection aligned with the granularity of concepts the practitioner seeks to audit.

Unlike manual approaches requiring domain experts to curate example images for each concept, ACE builds a concept bank without human intervention. This automation scales to domains with thousands of potential concepts and eliminates annotator bias, though discovered concepts still require human review for semantic labeling and interpretability validation.