Free-Response Operating Characteristic (FROC)

A related statistical curve that evaluates detection and localization accuracy when only one target is present per image. Unlike FROC, which handles multiple lesions, LROC plots the probability of correct localization against the false positive rate for single-target cases.

• Used in simpler detection tasks with a single abnormality per image
• The x-axis represents false positive rate (not average false positives per image)
• Often serves as a bridge between classic ROC and FROC analysis
• Less suitable for mammography where multiple calcification clusters may coexist

A variant of FROC analysis where the x-axis is the false positive fraction rather than the average number of false positives per image. This normalizes the false positive axis to a 0-1 scale, making it easier to compare studies with different numbers of images.

• The AFROC curve always ends at (1,1) by definition
• Area under the AFROC curve provides a single summary metric
• Useful when comparing systems across datasets of varying sizes
• Sacrifices the intuitive 'false positives per image' interpretation

The spatial rule defining when an AI-generated mark is considered a true positive localization. Common criteria include:

• Center-to-center distance: The Euclidean distance between the mark center and the lesion centroid must be below a threshold (e.g., 50 pixels)
• Overlap ratio: The intersection-over-union (IoU) between the marked region and the ground truth bounding box must exceed a minimum (e.g., 0.3)
• Distance-to-boundary: For mammography, the mark must fall within a specified radius from the lesion margin

These criteria directly impact FROC curve shape and must be predefined in the study protocol.

A summary scalar derived from FROC data, commonly used in grand challenges such as the LUNA16 lung nodule detection challenge. The CPM averages sensitivity values at predefined false positive rate thresholds.

• Typical thresholds: 0.125, 0.25, 0.5, 1, 2, 4, and 8 false positives per scan
• Provides a single number for leaderboard ranking
• Heavily weights performance in the clinically relevant low-FP region
• Criticized for being sensitive to the chosen threshold set

The process of defining the reference standard against which FROC curves are computed. In mammography CADe, this typically involves:

• Consensus panel: Multiple expert radiologists independently annotate lesions, with arbitration for disagreements
• Histopathological confirmation: Biopsy-proven malignancies serve as definitive ground truth
• Interval cancer tracking: Cancers diagnosed within 12-24 months of a negative screen are retrospectively localized

Incomplete or erroneous ground truth leads to misclassified marks and distorted FROC curves.