Weakly Supervised Object Detection

WSOD models learn from binary or multi-class labels assigned to entire images, not individual regions. The model must autonomously discover which pixels constitute the object of interest.

• Input: A chest X-ray labeled 'pneumonia present'
• No bounding boxes: The model never sees explicit location coordinates during training
• Self-discovery: The network learns to identify discriminative regions through iterative refinement
• Key mechanism: Multiple Instance Learning (MIL) treats each image as a bag of region proposals, where at least one proposal must contain the target class

WSOD is fundamentally built on Multiple Instance Learning (MIL), where images are 'bags' containing many region proposals (instances). The bag is labeled positive if at least one instance contains the object.

• Positive bag: Contains at least one true lesion region among hundreds of proposals
• Negative bag: All proposals are background tissue
• MIL pooling: Aggregates instance-level scores into an image-level prediction
• Challenge: The model must identify which instance triggered the positive label without explicit guidance

WSOD models progressively improve localization through self-training loops. Initial coarse activations are refined over multiple training epochs.

• Phase 1: The model identifies broad, diffuse regions of interest using class activation maps
• Phase 2: High-confidence proposals from Phase 1 serve as pseudo-ground-truth for subsequent training
• Phase 3: The detector tightens bounding boxes around the most discriminative parts
• Common pitfall: Models often focus on the most distinctive part rather than the entire object (e.g., a tumor's calcified core instead of its full extent)

Class Activation Maps (CAMs) are the primary mechanism WSOD models use to generate initial spatial localization from image-level labels. CAMs highlight which image regions most influenced the classification decision.

• Grad-CAM: Uses gradient flow into the final convolutional layer to produce coarse heatmaps
• CAM variants: Grad-CAM++, Score-CAM, and Ablation-CAM improve localization precision
• Limitation: CAMs produce low-resolution maps that rarely capture precise object boundaries
• Medical application: Identifies approximate tumor locations in whole-slide pathology images using only a 'malignant' label

WSOD trades localization precision for dramatically reduced annotation costs. This trade-off is particularly valuable in medical imaging where expert radiologist time is scarce.

• Full supervision: Requires pixel-perfect bounding boxes — 30-60 seconds per lesion for a radiologist
• Weak supervision: Requires only a binary label — < 1 second per image
• Accuracy gap: WSOD typically achieves 70-85% of fully-supervised mAP on standard benchmarks
• Best use case: Initial screening and triage where approximate localization is sufficient to flag studies for expert review

WSOD excels in high-volume screening scenarios where the goal is to identify potentially abnormal cases for prioritized radiologist review, not to provide final measurements.

• Tuberculosis screening: Flag chest X-rays with potential infiltrates using only image-level labels from radiology reports
• Mammography triage: Identify suspicious regions in screening mammograms to prioritize reading worklists
• Retinal imaging: Detect diabetic retinopathy lesions from fundus photographs labeled only with disease grade
• Integration pattern: WSOD model runs first; positive cases receive full annotation or immediate expert review

The foundational mathematical framework for weak supervision. In MIL, the model is trained on bags of instances (e.g., patches from a whole slide image) where only the bag-level label is known. A positive bag contains at least one positive instance, while a negative bag contains none. The network learns to score individual instances and aggregate them via a permutation-invariant pooling operator, such as noisy-OR or attention-based MIL pooling, to produce a bag-level prediction. This enables the model to implicitly localize lesions without pixel-level annotations.

A pivotal technique for generating localization maps from image-level labels. CAM leverages the global average pooling layer in a CNN to project the weights of the output layer back onto the final convolutional feature maps. The resulting class activation map highlights the discriminative image regions used for classification. Weakly supervised detectors often use CAM variants like Grad-CAM++ or Score-CAM as pseudo-ground truth to train downstream bounding box regressors, effectively bridging the gap between classification and detection.

A self-training strategy where a model trained on image-level labels generates noisy bounding box proposals, which are then iteratively refined. The process typically involves:

• Seed Generation: Using CAM or MIL to produce initial coarse localization maps.
• CRF Refinement: Applying a Conditional Random Field to sharpen boundaries and enforce spatial continuity.
• Iterative Retraining: Treating the refined masks as ground truth to train a fully supervised detector like Faster R-CNN, repeating the process to bootstrap performance. This closes the gap with fully supervised methods.

A middle ground between image-level and bounding box annotation. Instead of drawing full boxes, annotators simply click on the center of each lesion. A single point annotation per object drastically reduces labeling time while providing crucial spatial cues. Weakly supervised models can incorporate these points as objectness priors or use them to train a center-point detector. This approach is highly practical for radiology, where a radiologist can mark a nodule with a single click in seconds.

Standard object detection metrics apply, but with specific considerations for weak supervision. mAP (mean Average Precision) at IoU 0.5 is the primary metric. However, weakly supervised models often struggle with precise boundary delineation, so CorLoc (Correct Localization)—which measures the percentage of images where the highest-scoring proposal correctly localizes the object—is also reported. For medical tasks, Free-Response ROC (FROC) is critical, as it evaluates sensitivity against the average number of false positives per scan, directly measuring clinical utility.

These two paradigms are often conflated but address different data regimes. Weakly Supervised Object Detection (WSOD) uses only image-level labels for the entire dataset—no bounding boxes exist. Semi-Supervised Object Detection (SSOD) assumes a small set of fully annotated images (with boxes) and a large set of unlabeled images. SSOD leverages the labeled subset to guide learning on the unlabeled data, often via consistency regularization or pseudo-labeling, and typically achieves higher precision than pure WSOD.