Quantus

Measures how accurately an explanation reflects the model's true reasoning process. Faithfulness metrics perturb inputs based on attribution scores and measure the resulting change in model output.

• Pixel Flipping: Iteratively removes the most salient pixels and plots the degradation in classification confidence
• Region Perturbation: Replaces highly attributed image regions with random noise or a baseline value
• Selectivity: Quantifies how quickly the model's prediction drops as the most relevant features are removed
• IROF (Iterative Removal of Features): Evaluates faithfulness by segmenting the input and removing segments in order of decreasing attribution

Evaluates the stability of explanations under small, semantically meaningless perturbations to the input. A robust explanation should not change dramatically when imperceptible noise is added.

• Local Lipschitz Estimate: Computes the maximum rate of change of the explanation under bounded input perturbations
• Continuity: Measures how smoothly the attribution changes as the input is continuously deformed
• Max-Sensitivity: Finds the maximum difference in explanations for inputs within a small epsilon-neighborhood
• Relative Input Stability: Compares the explanation's sensitivity to the model's own output sensitivity

Quantifies the conciseness and human-interpretability of an explanation. Effective explanations should be sparse and avoid scattering attribution across irrelevant features.

• Sparseness: Uses the Gini index or entropy to measure how concentrated the attribution mass is across input features
• Effective Complexity: Counts the number of features required to explain a given fraction of the total attribution
• Complexity-Sensitivity Trade-off: Evaluates whether an explanation achieves high faithfulness with minimal feature complexity
• Smoothness: Measures spatial coherence, penalizing fragmented or noisy saliency maps that are difficult for clinicians to interpret

Applies sanity checks that any valid explanation method must pass, ensuring the attribution is genuinely dependent on the model's learned parameters and not an artifact of the input data.

• Model Parameter Randomisation Test: Cascades randomisation of model layers from top to bottom; a valid explanation should degrade progressively
• Data Randomisation Test: Trains the model on permuted labels; the resulting explanations should differ fundamentally from those of a properly trained model
• Completeness: Verifies that the sum of all feature attributions equals the difference between the model output at the input and a baseline
• Implementation Invariance: Confirms that functionally equivalent networks produce identical attributions

Evaluates how precisely an explanation's high-attribution regions correspond to known ground-truth bounding boxes or segmentation masks, critical for object detection and medical lesion attribution.

• Pointing Game: Measures the fraction of highest-attribution points that fall within the ground-truth object bounding box
• Top-k Intersection: Calculates the overlap between the k most salient pixels and the annotated region of interest
• Relevance Mass Accuracy: Computes the proportion of total attribution that falls inside the target region versus the background
• Energy-Based Pointing Game: Extends the pointing game with distance-weighted scoring for more nuanced localisation assessment

Provides a consistent API and standardised benchmarking protocol for comparing explanation methods across models, datasets, and tasks. Eliminates ad-hoc evaluation scripts and enables reproducible research.

• Metric Taxonomy: Organises 30+ metrics into well-defined categories (faithfulness, robustness, complexity, localisation, randomisation)
• Batch Processing: Evaluates explanations across entire datasets with configurable perturbation parameters and baseline strategies
• Normalisation & Aggregation: Standardises metric outputs to a common scale and provides summary statistics for clean reporting
• Extensible Architecture: Allows custom metrics to be registered and integrated into the evaluation pipeline alongside built-in metrics

A core metric category within Quantus that measures how accurately an explanation reflects the model's true reasoning. A faithful saliency map will show a significant drop in model confidence when the highlighted regions are perturbed or removed.

• Monotonicity: Higher attribution must correlate with higher impact
• Pixel Flipping: Measures area under the curve when removing most salient pixels
• Selectivity: Evaluates how quickly the model's prediction changes

Quantus provides metrics to test whether an explanation remains stable under small, imperceptible perturbations to the input. A robust explanation should not change dramatically when minor noise is added to a medical scan.

• Local Lipschitz Estimate: Measures maximum explanation change under bounded input perturbation
• Continuity: Evaluates explanation consistency in the input neighborhood
• Critical for ensuring regulatory explainability in diagnostic AI

Quantus quantifies the human-interpretability of an explanation by measuring its complexity. A saliency map that highlights a sparse, coherent region is more clinically useful than a noisy, fragmented heatmap.

• Sparseness: Measures via the Gini index of attribution values
• Effective Complexity: Counts the number of contiguous, high-attribution regions
• Smoothness: Evaluates attribution variance between neighboring pixels

A security vulnerability where an adversary manipulates an input to produce a misleading explanation while maintaining the original classification. Quantus can be used to benchmark explanation methods against such attacks.

• The model correctly classifies a tumor, but the saliency map points to healthy tissue
• Undermines trust calibration between clinicians and AI
• Quantus's robustness metrics help quantify susceptibility to these attacks

The false sense of security that arises from viewing a plausible-looking but unfaithful saliency map. Quantus directly combats this by providing quantitative, falsifiable metrics that expose explanations that look convincing but do not reflect the model's true decision process.

• A smooth, anatomically-plausible heatmap may still be unfaithful
• Quantus's faithfulness and randomisation tests reveal these illusions
• Essential for SaMD audit trail integrity