Mean Average Precision (mAP)

The traditional Pascal VOC metric that computes mAP at a single Intersection over Union (IoU) threshold of 0.5. A predicted bounding box is considered a true positive if its IoU with the ground truth box is ≥ 50%.

• Use case: Rapid prototyping and initial model comparison
• Limitation: Does not reward highly precise localization; a box with 0.51 IoU and one with 0.95 IoU are treated identically
• Legacy status: Still reported for historical comparison but considered lenient

The primary COCO evaluation metric that averages mAP across ten IoU thresholds from 0.5 to 0.95 in 0.05 increments. This provides a comprehensive view of detection performance across all localization qualities.

• Calculation: mAP at each threshold is computed independently, then averaged
• Advantage: Rewards models that produce tight, precise bounding boxes
• Industry standard: The default metric reported in most modern object detection papers and benchmarks

A strict variant that requires an IoU of 0.75 or higher for a positive match. This metric specifically evaluates a model's ability to produce highly precise bounding box localizations.

• Relevance: Critical for applications requiring exact spatial measurements, such as metrology and robotic grasping
• Trade-off: High [email protected] indicates excellent localization but may come at the cost of recall if the model is overly conservative
• Reporting: Often presented alongside [email protected] to show the performance drop under stricter criteria

COCO evaluation breaks down mAP by object size to diagnose model bias toward specific scales. This is essential for manufacturing inspection where defect sizes vary dramatically.

• Small objects: Area < 32² pixels — e.g., hairline cracks, microscopic solder defects
• Medium objects: 32² ≤ area < 96² pixels — e.g., surface dents, label misprints
• Large objects: Area ≥ 96² pixels — e.g., assembly errors, missing components
• Diagnostic value: A high overall mAP masking poor small-object performance is a common failure mode in quality inspection systems

While standard mAP gives equal weight to every class, weighted mAP accounts for class imbalance by weighting each class's Average Precision by its frequency in the dataset or its operational importance.

• Per-Class AP: The AP computed independently for each object class, revealing which defect types the model struggles with
• Class imbalance problem: A model achieving high mAP by excelling at frequent classes while failing on rare but critical defects (e.g., safety-critical anomalies) is dangerously misleading
• Weighting strategies: Frequency-based (support-weighted), cost-based (weighted by scrap cost or recall priority), or uniform (standard mAP)

Measures the exactness of a model's positive predictions. It answers: Of all the products the model flagged as defective, how many were actually defective?

• Formula: True Positives / (True Positives + False Positives)
• A high precision score minimizes costly false alarms that would send good products to the scrap bin.
• In manufacturing, low precision directly increases the False Reject Rate (FRR) , wasting materials and operator time.

Measures the completeness of a model's positive predictions. It answers: Of all the truly defective products on the line, how many did the model catch?

• Formula: True Positives / (True Positives + False Negatives)
• A high recall score minimizes dangerous misses that let defective products ship to customers.
• In quality assurance, low recall directly increases the Escape Rate, the most critical safety and reputation metric.

A graphical plot that visualizes the trade-off between precision and recall at every possible classification threshold. As you lower the confidence threshold to catch more defects (higher recall), you inevitably misclassify more good parts (lower precision).

• The curve typically has a zigzag downward slope.
• The area under this curve is the Average Precision (AP) for a single class.
• A model that maintains high precision even at high recall levels is considered robust.

The metric that determines whether a predicted bounding box counts as a true positive or a false positive. It quantifies the spatial overlap between the predicted box and the ground truth annotation.

• Formula: Area of Overlap / Area of Union
• An IoU threshold (e.g., 0.50) is set to define a correct detection. [email protected] and [email protected] are common variants.
• A higher IoU threshold demands tighter, more precise localization, which is critical for metrology and precise robotic picking.

The summary metric for a single object class. It condenses the information in the Precision-Recall curve into a single scalar value by calculating the area under the curve.

• Before calculating the area, the raw precision-recall curve is often interpolated to reduce the impact of small zigzags.
• AP is calculated independently for each defect class, such as 'scratch,' 'dent,' or 'crack.'
• It provides a single, threshold-independent score for class-specific detection performance.

A legacy method for summarizing the Precision-Recall curve, primarily used in the PASCAL VOC challenge. It samples precision at 11 equally spaced recall levels (0.0, 0.1, ..., 1.0).

• For each recall level, it takes the maximum precision value at that recall level or any higher recall level.
• The final AP is the simple average of these 11 maximum precision values.
• Modern evaluation, like the COCO challenge, uses a more precise all-point interpolation, averaging precision at 101 recall points.