Weighted Boxes Fusion (WBF)

Unlike Non-Maximum Suppression (NMS), which discards overlapping boxes, WBF constructs a fused box by averaging the coordinates of all proposed boxes from every model in the ensemble.

• The fused box's coordinates are a weighted sum based on each model's confidence score.
• This retains information from multiple models, even if one model's box is slightly offset.
• Example: If Model A predicts a lesion at (10,10,50,50) with 0.9 confidence and Model B predicts (12,8,48,52) with 0.8 confidence, WBF merges them into a single, averaged box rather than choosing one.

WBF generates a new, fused confidence score for the averaged bounding box based on the collective agreement of the ensemble.

• The final confidence is calculated as the average of the individual model confidences, weighted by the number of models that predicted the box.
• This penalizes detections made by only a single model and rewards consensus.
• Formula: C = (Σ confidence_i * w_i) / N where N is the total number of models and w_i is a per-model weight factor.

WBF uses an Intersection over Union (IoU) threshold to determine which boxes from different models refer to the same object before fusing them.

• Boxes with an IoU above the threshold are matched into a cluster and fused together.
• Boxes below the threshold are treated as separate detections.
• This parameter directly controls the algorithm's sensitivity to distinguishing adjacent objects, such as two closely located nodules.

WBF consistently outperforms standard NMS and Soft-NMS when fusing predictions from an ensemble of diverse detection architectures.

• NMS selects a single best box, discarding potentially valuable localization information from other models.
• WBF is particularly effective when models have different strengths (e.g., one model is better at scale, another at localization).
• Benchmark: In medical imaging challenges, WBF has been shown to improve mAP (mean Average Precision) by 1-3% over traditional NMS-based ensembling.

WBF allows for the assignment of different weights to each model in the ensemble, reflecting their individual reliability.

• A more accurate model can be given a higher weight, causing its coordinates to dominate the fused box.
• Weights can be derived from validation set performance or domain-specific heuristics.
• Example: A Faster R-CNN model trained specifically on small lesions might be weighted higher than a general YOLO model when fusing predictions for micro-calcification detection.

WBF is a critical tool for clinical AI pipelines where minimizing false negatives and maximizing localization precision is paramount.

• It is used to ensemble models trained on different data folds, augmentations, or architectures for robust lesion localization.
• By averaging predictions, WBF reduces the variance of the ensemble, leading to more stable and trustworthy bounding boxes for radiologist review.
• It integrates seamlessly with Test Time Augmentation (TTA) workflows, fusing predictions from multiple augmented versions of the same image.

The standard post-processing algorithm that WBF often replaces. NMS operates by selecting the highest-confidence box and suppressing all other boxes with an IoU above a threshold.

• Greedy algorithm: Iteratively selects and removes
• Weakness: Discards potentially useful information from overlapping predictions
• Soft-NMS variant: Decays scores instead of hard removal, but still doesn't fuse boxes

WBF improves on NMS by averaging all predictions rather than discarding them, preserving the collective intelligence of multiple models.

The foundational metric that determines which boxes get fused in WBF. IoU measures the overlap ratio between two bounding boxes, calculated as area of intersection divided by area of union.

• IoU Threshold: WBF uses this to determine which boxes belong to the same object cluster
• Typical value: 0.55 for WBF clustering, lower than standard NMS thresholds
• Fusion weight: Boxes with higher IoU to the cluster mean receive proportionally greater weight

Without precise IoU calculation, WBF cannot correctly group predictions from different models.

A complementary ensembling strategy that pairs powerfully with WBF. TTA applies multiple augmentations (flips, rotations, scales) to a single test image and runs detection on each variant.

• Workflow: Image → Augmentations → Model → WBF fusion of all variants
• Benefit: Produces diverse predictions from a single model
• Synergy: WBF merges TTA predictions more effectively than NMS by averaging coordinates across augmented views

Combining TTA with multi-model WBF creates a two-tier ensembling strategy that maximizes robustness.

The reliability of WBF's weighted averaging depends directly on well-calibrated confidence scores. If a model's confidence estimates are poorly calibrated, its predictions will distort the fused result.

• Calibration methods: Platt scaling, isotonic regression, temperature scaling
• Impact on WBF: Overconfident models dominate the weighted average incorrectly
• Best practice: Calibrate each model independently before ensembling with WBF

A model outputting 0.99 confidence for a false positive will pull the fused box toward an incorrect location if its weight isn't properly calibrated.

The standard evaluation metric used to quantify WBF's improvement over baseline methods. mAP computes the area under the precision-recall curve averaged across all classes and IoU thresholds.

• COCO mAP: Evaluates at IoU thresholds from 0.50 to 0.95 in 0.05 increments
• WBF impact: Typically improves mAP by 0.5-2.0 points over single-model NMS baselines
• Trade-off: WBF may slightly reduce precision at very high IoU thresholds while boosting recall

Always evaluate WBF against both NMS and Soft-NMS baselines using the same mAP protocol to quantify the true improvement.

WBF yields the greatest gains when the ensembled models make different types of errors. Homogeneous models produce correlated predictions that don't benefit from fusion.

• Architecture diversity: Combine YOLO, Faster R-CNN, and DETR
• Training diversity: Different backbones, input resolutions, or data augmentations
• Checkpoint diversity: Fuse predictions from different training epochs
• Scale diversity: Models trained on different input scales detect different object sizes

A WBF ensemble of three identical models trained with the same seed provides negligible improvement over a single model.