Weakly Supervised Segmentation

The method leverages a spectrum of weak labels, each providing less spatial information than a full pixel mask:

• Image-level tags: Binary or multi-class labels indicating presence of a pathology.
• Bounding boxes: Coarse localization rectangles drawn around objects.
• Scribbles: Sparse lines drawn inside and outside regions of interest.
• Points: Single clicks on object centers or extreme points. The model must infer the full extent of the structure from these hints.

A foundational technique that repurposes image classifiers for localization. Global Average Pooling is applied to the final convolutional feature maps, and the weights of the classification layer are projected back onto the feature maps to generate a coarse heatmap. This heatmap highlights discriminative regions used by the network to make its classification decision, serving as a pseudo-segmentation mask without any pixel-level training data.

Training relies on specialized loss functions that enforce consistency with weak labels rather than direct pixel comparison:

• Size constraints: Penalize predictions where the segmented area deviates from a known expected size derived from bounding boxes.
• CRF Loss: A fully-connected Conditional Random Field is applied as a recurrent layer to enforce spatial smoothness and boundary alignment without ground truth masks.
• Partial Cross-Entropy: Loss is computed only on annotated scribble pixels, ignoring unlabeled regions.

A self-training paradigm where an initial model trained on weak labels generates pseudo-masks for the full dataset. These noisy masks are then treated as ground truth to retrain a more robust model. Advanced approaches use co-training with multiple networks that teach each other, or expectation-maximization to iteratively refine the latent segmentation variable and model parameters until convergence.

In radiology workflows, drawing a single bounding box around a Gross Tumor Volume (GTV) takes approximately 10-15 seconds, while a full pixel-level contour requires 5-20 minutes per slice. Weakly supervised methods trained on bounding boxes can achieve Dice scores within 5-10% of fully supervised models, dramatically reducing the time and cost of curating training datasets for Organ-at-Risk (OAR) segmentation.

A paradigm where images are treated as bags of patches, and only the bag-level label is known. The network learns to identify which patches within the bag are positive. In Whole Slide Image analysis, a slide labeled 'cancerous' contains millions of patches; MIL identifies the specific malignant regions without requiring a pathologist to annotate every pixel, making gigapixel segmentation computationally tractable.

Weakly supervised segmentation reduces annotation cost by using coarse labels instead of dense pixel masks. Common annotation types include:

• Image-level tags: Binary or multi-class labels indicating the presence or absence of a pathology without spatial information.
• Bounding boxes: Tight rectangular regions enclosing the object of interest, providing rough localization.
• Scribbles: Sparse, hand-drawn lines on the foreground and background, offering minimal but precise spatial guidance.
• Points: Single clicks on the object center or extreme points, the most minimal form of spatial supervision. These weak labels are used to derive proxy supervisory signals, such as class activation maps (CAMs), to train a full segmentation model.

A foundational technique for localizing objects using only image-level labels. CAM leverages the global average pooling layer in a classification CNN to identify the discriminative image regions used for a prediction.

• The final convolutional feature maps are weighted by the learned importance of each channel for a specific class.
• A linear combination of these weighted maps produces a coarse heatmap highlighting the object's approximate location.
• Grad-CAM extends this to any CNN architecture by using gradient signals flowing into the final convolutional layer. These heatmaps serve as pseudo-ground truth masks to train a downstream segmentation network.

A training paradigm where the model learns from bags of instances rather than individually labeled samples. In segmentation, an entire image is a 'bag' of pixels or patches.

• In a positive bag, at least one instance belongs to the target class (e.g., contains a tumor).
• In a negative bag, no instances belong to the target class. The model learns to identify which instances within a positive bag are responsible for the bag-level label, effectively performing localization without pixel-level annotations. This is particularly powerful for whole slide image analysis in pathology.

Pseudo-masks generated from weak labels are often noisy and lack well-defined boundaries. Conditional Random Fields (CRFs) are a classic post-processing step to refine these coarse predictions.

• CRFs model the relationship between every pixel pair in an image as a fully connected graph.
• The energy function encourages label agreement between pixels that are spatially close and have similar color/intensity values.
• This smooths the segmentation while preserving sharp edges at intensity boundaries. Integrating a CRF as a differentiable recurrent neural network (as in CRF-RNN) allows for end-to-end training, jointly optimizing the segmentation network and the boundary refinement process.

The performance of weakly supervised methods is benchmarked against fully supervised models using standard segmentation metrics, though a performance gap is expected.

• Intersection over Union (IoU): Measures the overlap between the predicted mask and the ground truth. A critical metric for evaluating localization accuracy from weak labels.
• Dice Score (F1 Score): The harmonic mean of precision and recall at the pixel level, sensitive to both false positives and false negatives.
• Hausdorff Distance: Quantifies the maximum boundary error, revealing if weak supervision fails to capture fine structural details. The goal is to minimize the gap between weakly supervised and fully supervised performance, often expressed as a percentage of the fully supervised baseline.

A specific weak supervision strategy where the model learns to propagate sparse scribble annotations into a dense segmentation mask.

• A common approach uses a partial cross-entropy loss, which is computed only on the annotated scribble pixels and ignored elsewhere.
• To prevent the model from predicting only the scribbles, a normalized cut loss or conditional random field loss is added as a regularizer. This encourages the segmentation to follow visual contours and produce spatially coherent, solid object masks.
• This method provides a direct, intuitive way for annotators to guide the model with minimal effort, often achieving results close to full supervision on organs with clear boundaries.