mAP (mean Average Precision)

The primary COCO evaluation metric that averages mAP across IoU thresholds from 0.50 to 0.95 in 0.05 increments. This rewards models with precise localization, not just rough detection.

• Clinical relevance: Distinguishes between a model that roughly localizes a tumor versus one that precisely delineates its boundaries for surgical planning.
• Calculation: mAP = mean([email protected], [email protected], ..., [email protected])
• Typical baseline: State-of-the-art medical detectors often achieve 0.45-0.60 mAP@[.5:.95] on challenging multi-class lesion benchmarks.

A stratified mAP variant that isolates performance on objects occupying less than 32×32 pixels in the source image. Critical for evaluating detection of micro-calcifications, early-stage nodules, and distal emboli.

• Why standard mAP fails: A model with high overall mAP may completely miss small pathologies if it excels only on large, obvious masses.
• Implementation: Filter ground-truth annotations by bounding box area before computing AP.
• Benchmark example: The DeepLesion dataset reports mAP_s separately to highlight small lesion detection capability.

Adapts mAP to the Free-Response Operating Characteristic (FROC) paradigm, where the metric is computed at clinically acceptable false-positive rates rather than IoU thresholds alone.

• Mechanism: Computes sensitivity at predefined false-positive intervals (e.g., 0.125, 0.25, 0.5, 1, 2, 4, 8 FPs per scan) and averages them.
• Clinical rationale: Radiologists tolerate a certain number of false marks per case; the metric must reflect this operational constraint.
• Standard in: LUNA16 lung nodule detection challenge and many FDA clearance studies.

Aggregates detection results at the patient level rather than the slice level, preventing volume-based bias where patients with more slices disproportionately influence the metric.

• Problem solved: A patient with 500 CT slices contributes 500× more to slice-level mAP than a patient with 50 slices, skewing evaluation.
• Method: Merge all detections for a single patient across their volumetric scan, apply NMS across slices, then compute AP per patient.
• Critical for: Multi-slice CT and MRI studies where lesion presence is a binary per-patient clinical decision.

Computes mAP by giving equal weight to each pathology class regardless of its frequency in the dataset, preventing common classes from dominating the metric.

• Standard mAP flaw: Macro-averaging across images means a class appearing in 90% of images dominates the score.
• Implementation: Compute AP independently per class, then take the unweighted mean: mAP_cb = (1/C) * Σ AP_c
• Use case: Evaluating detection of rare pathologies (e.g., pneumothorax, rib fracture) alongside common ones (e.g., nodule, consolidation) in chest X-ray benchmarks like VinBigData.

The legacy PASCAL VOC metric using a single IoU threshold of 0.50. Still widely reported in medical imaging literature for comparability with older baselines.

• Limitation: Rewards loose bounding boxes that may be clinically inadequate for radiation therapy planning or volumetric measurement.
• Continued use: Many medical grand challenges (e.g., early MICCAI competitions) established baselines at [email protected], making it necessary for historical comparison.
• Modern practice: Report alongside mAP@[.5:.95] to show both rough and precise localization capability.

The foundational geometric metric that quantifies the spatial overlap between a predicted bounding box and a ground truth annotation. Calculated as the area of overlap divided by the area of union, IoU determines whether a detection is a true positive or false positive. In medical imaging, an IoU threshold of 0.5 to 0.75 is typical, balancing localization strictness for small lesions against acceptable clinical variance.

• Formula: IoU = (Area of Overlap) / (Area of Union)
• Range: 0.0 (no overlap) to 1.0 (perfect match)
• Clinical relevance: A low IoU on a tumor boundary can misrepresent volumetric measurements critical for RECIST criteria

The graphical plot from which Average Precision (AP) is derived. The curve illustrates the trade-off between precision (the fraction of detections that are correct) and recall (the fraction of ground truth objects found) at every possible confidence threshold. A detection model with a curve that hugs the top-right corner demonstrates both high sensitivity and a low false alarm rate.

• Precision: TP / (TP + FP)
• Recall: TP / (TP + FN)
• Area Under Curve: The literal definition of Average Precision for a single class

Two distinct methods for calculating the area under the Precision-Recall curve. The legacy 11-point interpolation samples precision at 11 equally spaced recall levels (0.0, 0.1, ..., 1.0), smoothing the curve. The modern all-point interpolation standard, used in the PASCAL VOC 2010+ and MS COCO benchmarks, interpolates precision at every unique recall level, providing a more accurate and less optimistic measure of true detection capability.

• Legacy method: Averages 11 fixed points
• Modern method: Integrates over all recall points
• Impact: All-point interpolation penalizes erratic precision drops more heavily

The de facto standard evaluation protocol that computes mAP by averaging AP across 10 IoU thresholds from 0.50 to 0.95 in 0.05 increments. This composite metric, denoted as mAP@[.50:.95], rewards models that localize objects with high spatial fidelity, not just those that find them. For medical object detection, this protocol is increasingly adopted to ensure precise lesion boundary delineation.

• Primary metric: mAP@[.50:.95] (averaged over 10 IoU thresholds)
• Secondary metrics: [email protected] (lenient), [email protected] (strict)
• Small/Medium/Large: mAP computed separately for objects of different pixel areas

A critical post-processing step that directly impacts mAP by removing redundant, overlapping detections for the same object. NMS sorts all predicted boxes by confidence score, selects the highest-scoring box, and suppresses any other box with an IoU exceeding a set threshold. Aggressive NMS can increase precision but lower recall by eliminating true adjacent lesions; lenient NMS can inflate false positives.

• Key parameter: IoU suppression threshold (commonly 0.3–0.5)
• Soft-NMS: Decays scores of overlapping boxes instead of hard-deleting them
• Clinical risk: Overly aggressive NMS can suppress a second, distinct lesion located near a primary finding

An alternative evaluation metric to mAP specifically designed for medical detection tasks where an image may contain an arbitrary number of lesions. FROC plots sensitivity (recall) against the average number of false positives per image, rather than against precision. This metric is the standard for FDA clearance submissions for Computer-Aided Detection (CADe) devices, as it directly quantifies the clinical burden of false alarms a radiologist must dismiss.

• X-axis: False positives per scan (e.g., 0.125, 0.25, 0.5, 1, 2, 4, 8)
• Y-axis: Lesion-level sensitivity
• Regulatory context: A defined operating point on the FROC curve is often required for clinical validation