Mask R-CNN

Mask R-CNN inherits the Region Proposal Network (RPN) from Faster R-CNN, a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN slides over the feature map generated by a backbone like ResNet combined with a Feature Pyramid Network (FPN) to handle objects at multiple scales. This two-stage approach—propose, then classify—provides the high detection accuracy required for clinical applications where missing a small lesion is unacceptable.

The defining innovation is a third output branch added to the existing classification and bounding-box regression heads. This Fully Convolutional Network (FCN) branch operates on each Region of Interest (RoI) and predicts a binary mask for each class independently, avoiding competition between classes. Critically, this decouples classification from segmentation: the class is determined by the classification branch, and the mask branch simply generates a mask for that class. This design yields superior instance segmentation compared to prior methods that entangled the two tasks.

A critical architectural fix over its predecessor, RoIPool. RoIPool performs harsh quantization by rounding floating-point coordinates to integers when extracting features for each RoI, introducing misalignments that degrade mask predictions. RoIAlign removes this quantization entirely by using bilinear interpolation at four regularly sampled locations in each pooling bin. This preserves exact spatial locations and improves mask accuracy by 10-50% relative, making it essential for medical image segmentation where boundary precision is clinically significant.

Training optimizes a combined loss: L = L_cls + L_box + L_mask. The classification loss (L_cls) and bounding-box regression loss (L_box) are identical to Faster R-CNN. The mask loss (L_mask) is the average binary cross-entropy loss applied only to the ground-truth class's mask, computed per-pixel on the predicted K masks (one per class). This per-pixel sigmoid and binary loss allows each class's mask to be generated without competition, a design choice that significantly improves instance segmentation performance over softmax-based approaches.

Mask R-CNN leverages an FPN backbone to build a multi-scale feature hierarchy with strong semantics at all levels. The FPN uses a top-down pathway with lateral connections to propagate semantically strong features from higher pyramid levels to lower, high-resolution levels. RoIs are assigned to different pyramid levels based on their scale using the formula: k = floor(k0 + log2(sqrt(wh)/224)). This scale-aware assignment ensures small anatomical structures are processed at high-resolution levels while large organs use coarser, semantically richer features.

Mask R-CNN performs instance segmentation, not semantic segmentation. In a chest X-ray with multiple nodules, semantic segmentation labels all nodule pixels as 'nodule' without distinguishing individual instances. Mask R-CNN detects each nodule separately, assigning a unique bounding box, class label, and pixel mask to each. This distinction is critical for clinical workflows requiring lesion counting, tracking individual tumor growth over time, or planning interventions on specific anatomical structures rather than amorphous tissue regions.