Non-Maximum Suppression (NMS)

NMS operates on a greedy selection principle to iteratively build a final set of detections. The algorithm sorts all predicted bounding boxes by their confidence score in descending order. It selects the highest-scoring box, then computes the Intersection over Union (IoU) between this box and all remaining boxes. Any box with an IoU exceeding a predefined threshold is considered a duplicate and is suppressed (removed). This process repeats with the next highest-scoring surviving box until no boxes remain. This deterministic behavior makes NMS highly predictable and debuggable in production environments.

• Sorting: Boxes are ranked by confidence score before processing.
• IoU Threshold: A hyperparameter, typically set between 0.3 and 0.7, that controls suppression aggressiveness.
• Iterative Loop: Continues until the candidate list is exhausted.

Without NMS, object detectors like YOLO or Faster R-CNN often produce numerous bounding boxes clustered around a single object. This happens because the model's region proposal network or grid-based prediction generates multiple high-confidence anchors for the same defect. NMS is the mechanism that collapses this cluster into a single, precise localization. For a quality assurance director, this means one defect on a part results in exactly one alarm, preventing a single scratch from triggering a cascade of false alerts that would halt the production line unnecessarily.

• Anchor Overlap: Inherent to dense prediction architectures.
• Single Alarm Principle: Ensures one defect equals one detection event.
• Clean Output: Prepares data for downstream metrics like Mean Average Precision (mAP).

Standard NMS uses a hard threshold, completely removing boxes above the IoU limit. This can be problematic in crowded scenes where two distinct objects of the same class are very close together. Soft-NMS addresses this by applying a continuous penalty function. Instead of zeroing out the score of an overlapping box, Soft-NMS decays its confidence score based on the IoU overlap. A box with 0.9 IoU receives a heavy penalty, while a box with 0.6 IoU receives a lighter one. This preserves detections of genuinely separate but adjacent objects, which is critical for inspecting densely packed components on a circuit board.

• Continuous Penalty: Uses a Gaussian or linear decay function.
• Crowded Scene Preservation: Retains detections for adjacent, non-duplicate objects.
• Computational Cost: Marginally higher than standard NMS due to score recalculation.

NMS can be applied in two modes depending on the detection architecture. Class-Agnostic NMS treats all bounding boxes equally regardless of their predicted class, suppressing any highly overlapping box. This is fast but risks suppressing a true positive of a different class that happens to overlap. Class-Specific NMS applies the suppression algorithm independently for each class. A 'scratch' detection will not suppress a 'dent' detection even if they overlap significantly. In manufacturing, class-specific NMS is essential when multiple defect types can co-occur on the same component surface.

• Class-Agnostic: Faster, but risks cross-class suppression.
• Class-Specific: Applies NMS per class label, preserving co-occurring defect types.
• Use Case: Essential for multi-class defect detection on complex assemblies.

The IoU threshold in NMS directly trades off between precision and recall. A low threshold (e.g., 0.3) aggressively suppresses boxes, reducing false positives and increasing precision, but risks removing true positives for clustered objects, lowering recall. A high threshold (e.g., 0.7) is more permissive, preserving true positives and boosting recall, but may allow duplicate detections to pass through, decreasing precision. Tuning this threshold on a validation set using a Gage Repeatability and Reproducibility (GR&R) study ensures the inspection system meets the required balance for the specific manufacturing tolerance.

• Low Threshold: Higher precision, lower recall. Risk of missed defects.
• High Threshold: Higher recall, lower precision. Risk of duplicate counts.
• Tuning: Must be validated against ground truth data for the specific product.

While NMS remains a standard post-processing step, some modern architectures aim to eliminate it. End-to-end detectors like the Detection Transformer (DETR) use a transformer encoder-decoder with a set-based global loss that performs bipartite matching, forcing the model to predict a unique set of detections without any need for NMS. Similarly, anchor-free methods like CenterNet predict object centers and regress dimensions, inherently reducing duplicate proposals. However, for high-speed, single-stage detectors like YOLO deployed on the edge, NMS remains the de facto standard due to its simplicity and low computational overhead.

• DETR: Uses bipartite matching loss to eliminate NMS entirely.
• CenterNet: Predicts a single heatmap peak per object.
• Edge Deployment: NMS remains preferred for its speed and deterministic execution on embedded hardware.

The primary geometric criterion that NMS uses to determine if two bounding boxes are detecting the same object. IoU computes the ratio of the area of overlap to the area of union between two boxes.

• Formula: IoU = Area of Overlap / Area of Union
• NMS Role: If the IoU between a high-confidence box and a lower-confidence box exceeds a set threshold (e.g., 0.5), the lower-confidence box is suppressed.
• Threshold Tuning: A higher IoU threshold (e.g., 0.7) is more permissive, keeping more boxes; a lower threshold (e.g., 0.3) is more aggressive, suppressing more.

The standard evaluation metric for object detectors that directly measures the impact of NMS on final model performance. mAP computes the area under the precision-recall curve averaged across all classes.

• Sensitivity to NMS: Overly aggressive NMS increases false negatives (missed detections), lowering recall and mAP. Overly permissive NMS increases false positives, lowering precision and mAP.
• [email protected]: The most common variant, using an IoU threshold of 0.5 to determine a true positive match.
• [email protected]:0.95: A stricter variant averaging mAP across IoU thresholds from 0.5 to 0.95 in 0.05 increments, penalizing imprecise localization.

The tabular foundation for understanding the downstream quality impact of NMS decisions. Every suppression or retention action shifts the counts of true positives, false positives, and false negatives.

• False Positive (FP): A redundant box that NMS failed to suppress, incorrectly flagged as a defect.
• False Negative (FN): A valid detection that NMS incorrectly suppressed, causing a missed defect.
• Precision/Recall Trade-off: Precision = TP / (TP + FP) and Recall = TP / (TP + FN). NMS threshold tuning directly navigates this trade-off.