Intersection over Union (Jaccard Index)

IoU is calculated as the area of overlap divided by the area of union between the predicted segmentation mask and the ground truth annotation.

• Formula: IoU = |A ∩ B| / |A ∪ B|
• Range: Bounded between 0.0 (no overlap) and 1.0 (perfect match)
• Pixel-Level: Operates on a per-pixel basis, treating each class independently
• Differentiable Surrogate: The Jaccard Loss (1 - IoU) is used directly as a training objective

IoU and the Dice Score (F1 Score) are monotonically related and convey the same ranking of segmentation quality, but differ in absolute value.

• Conversion: Dice = 2 * IoU / (1 + IoU)
• Penalty Profile: IoU always yields a lower numerical score than Dice for the same prediction, penalizing errors more severely
• Example: A Dice of 0.80 corresponds to an IoU of approximately 0.67
• Usage: IoU is standard in PASCAL VOC and COCO benchmarks; Dice is more common in medical imaging literature

In multi-class segmentation, IoU is computed per class and then aggregated to provide a holistic performance measure.

• Per-Class IoU: Calculates overlap independently for each semantic category (e.g., liver, tumor, kidney)
• Mean IoU (mIoU): The arithmetic average of IoU across all classes, the standard aggregate metric
• Weighted IoU: Averages per-class IoU weighted by each class's pixel frequency to account for class imbalance
• Background Dominance: mIoU prevents large background regions from masking poor performance on small, critical foreground objects

IoU requires discrete binary masks, necessitating a thresholding step when model outputs are continuous probability maps.

• Default Threshold: Probabilities ≥ 0.5 are typically assigned to the foreground class
• Threshold Sensitivity: The choice of threshold directly impacts IoU; optimizing it per-class can improve reported performance
• Soft IoU: A differentiable approximation that operates directly on continuous probabilities, used during training to avoid the non-differentiable thresholding step
• Calibration: Well-calibrated models produce probabilities that reflect true likelihood, making the 0.5 threshold semantically meaningful

While ubiquitous, IoU has known failure modes that motivate complementary metrics in clinical segmentation tasks.

• Boundary Insensitivity: IoU does not distinguish between a minor boundary shift and a complete miss on a small object; both yield zero overlap
• Size Bias: Small structures (e.g., a 10-pixel lesion) can have an outsized impact on per-class IoU due to high variance
• No Spatial Information: IoU is purely volumetric; it cannot capture the clinical acceptability of a boundary error
• Complementary Metrics: Use Hausdorff Distance to measure worst-case boundary deviation and Average Surface Distance for mean boundary error

In object detection tasks, IoU serves as the criterion for determining whether a predicted bounding box constitutes a true positive detection.

• Standard Threshold: An IoU ≥ 0.5 is the conventional threshold for a correct detection in the PASCAL VOC challenge
• Strict Threshold: COCO evaluation uses IoU ≥ 0.75 for its strict metric (AP@75)
• Average Precision (AP): Computed by averaging precision across IoU thresholds from 0.5 to 0.95 in 0.05 increments
• Clinical Relevance: For medical object detection (e.g., nodule localization), higher thresholds (≥0.7) are often required to ensure clinical utility