Region Proposal Network (RPN)

An RPN is implemented as a fully convolutional network (FCN), meaning it contains no fully connected layers. This design choice allows the network to accept input images of arbitrary size during both training and inference. A small network is slid over the convolutional feature map output by a backbone (e.g., ResNet), making it agnostic to the spatial dimensions of the input. This efficiency is critical for processing high-resolution medical images like mammograms or whole slide images.

At each sliding window location, the RPN simultaneously predicts multiple region proposals, parameterized relative to anchor boxes. These anchors are pre-defined bounding boxes of various scales and aspect ratios that serve as reference templates.

• Translation Invariance: The anchor mechanism ensures the same function predicts proposals regardless of location.
• Multi-Scale Design: A pyramid of anchors (e.g., 128², 256², 512² pixels with ratios 1:1, 1:2, 2:1) handles objects of different sizes, from small lung nodules to large tumors, without requiring multiple input scales.

The RPN has two sibling output layers for each anchor box:

• Classification (Objectness) Layer: A softmax classifier predicts the probability that an anchor contains an object versus being background. This is a binary classification task.
• Regression Layer: A bounding box regressor predicts the 4 coordinates of the proposal (x, y, width, height) as offsets relative to the anchor box. This dual-head design allows the network to simultaneously answer 'is there an object here?' and 'where exactly is it?' in a single forward pass.

The RPN is trained using a multi-task loss function that jointly optimizes classification and regression. Crucially, it uses a specific sampling strategy to handle extreme class imbalance:

• Positive Samples: Anchors with the highest Intersection over Union (IoU) overlap with a ground-truth box, or an IoU > 0.7.
• Negative Samples: Anchors with an IoU < 0.3 with all ground-truth boxes.
• Balanced Mini-Batch: A batch of 256 anchors is sampled with a 1:1 ratio of positives to negatives to prevent the loss from being dominated by easy background examples.

The RPN's defining innovation is that it shares the full-image convolutional features with the downstream detection network (Fast R-CNN). Instead of computing proposals on the raw image, the RPN operates on the feature map from the backbone. This sharing dramatically reduces the marginal cost of generating proposals to nearly zero, enabling near real-time detection speeds. This is a fundamental departure from predecessors like Selective Search, which ran as a separate, slow CPU-based algorithm.

Because the RPN generates proposals densely at every anchor location, it produces thousands of highly overlapping candidate boxes for a single object. To reduce redundancy, Non-Maximum Suppression (NMS) is applied based on the objectness scores. NMS iteratively selects the highest-scoring box and suppresses all others with an IoU overlap greater than a threshold (e.g., 0.7). The top-N proposals (e.g., 2000) are then passed to the downstream detector, ensuring a manageable and non-redundant set of high-quality regions.

A set of predefined bounding boxes with various scales and aspect ratios that serve as reference templates for the RPN. At each sliding window position, the network predicts offsets relative to these anchors rather than absolute coordinates.

• Typical scales: 128², 256², 512² pixels
• Common aspect ratios: 1:1, 1:2, 2:1
• A feature map of size W×H with k anchors generates W×H×k proposals

The evaluation metric that measures overlap between a predicted region proposal and a ground truth bounding box. The RPN uses IoU thresholds to assign binary class labels during training.

• Positive anchor: IoU > 0.7 with any ground truth box
• Negative anchor: IoU < 0.3 with all ground truth boxes
• Anchors with 0.3 ≤ IoU ≤ 0.7 are ignored during training

The technique that refines the coordinates of proposed region boxes. The RPN predicts four parameterized offsets (tx, ty, tw, th) relative to each anchor to improve localization accuracy.

• tx, ty: Scale-invariant translation of the box center
• tw, th: Log-space scaling of width and height
• Uses a smooth L1 loss to balance robustness against outliers

A post-processing algorithm applied to the RPN's output to eliminate redundant, highly overlapping proposals for the same object. Without NMS, thousands of near-identical boxes would flood the downstream detector.

• Ranks proposals by objectness score
• Suppresses boxes with IoU > threshold (typically 0.7)
• Retains only the top-N proposals (e.g., 2000) for the detection head

The two-stage detection framework that introduced the RPN. It shares full-image convolutional features between the proposal network and the detection network, enabling near real-time rates.

• Stage 1: RPN generates class-agnostic region proposals
• Stage 2: Fast R-CNN detector classifies proposals and refines boxes
• End-to-end training via 4-step alternating optimization
• Eliminated the external selective search bottleneck of Fast R-CNN

A multi-scale feature extractor that enhances the RPN by building a pyramidal hierarchy of feature maps. This allows the network to detect objects at vastly different sizes—critical for medical imaging where lesions range from tiny micro-calcifications to large masses.

• Bottom-up pathway: Standard convolutional feedforward
• Top-down pathway: Upsampled, semantically stronger features
• Lateral connections: 1×1 convolutions to merge pathways
• RPN heads are attached at each pyramid level independently