Cascade R-CNN

Cascade R-CNN chains multiple detection heads sequentially, where the output bounding boxes from one stage serve as the input proposals for the next. Each stage is trained with a progressively higher IoU threshold (e.g., 0.5, 0.6, 0.7), ensuring that each detector specializes in refining boxes of a specific quality level. This resampling mechanism progressively improves localization accuracy without the overfitting that occurs when a single detector is trained with a high threshold from the start.

A fundamental insight behind Cascade R-CNN is the distributional mismatch between training and inference. A detector trained at a low IoU threshold (e.g., 0.5) produces noisy, low-quality boxes. When these noisy boxes are fed to a detector trained at a high threshold (e.g., 0.7), performance degrades because the input distribution at inference differs from the training distribution. The cascade architecture solves this by sequentially lifting proposal quality to match each stage's training threshold.

Each cascade stage performs bounding box regression followed by resampling. The regressed boxes are used as proposals for the next stage, and the resampling operation ensures that the distribution of proposals shifts toward higher-quality examples. This iterative refinement is distinct from simply stacking detection heads—the resampling step actively filters and re-ranks proposals, creating a positive feedback loop that systematically improves localization.

Unlike Faster R-CNN, which uses a single IoU threshold for defining positives, Cascade R-CNN trains each stage with its own threshold:

• Stage 1: IoU > 0.5 — captures all potential objects, high recall
• Stage 2: IoU > 0.6 — filters false positives, improves precision
• Stage 3: IoU > 0.7 — final refinement for high-quality localization This staged curriculum prevents the overfitting that occurs when a single detector is forced to handle the full spectrum of proposal quality.

During inference, an image passes through the Region Proposal Network (RPN) to generate initial proposals. These proposals flow through the cascade:

• Stage 1 classifies and regresses all proposals at IoU 0.5
• Regressed boxes are resampled and passed to Stage 2 at IoU 0.6
• Stage 3 produces the final detections at IoU 0.7 Each stage uses the same RoI Align features from a shared backbone, making the cascade computationally efficient despite the sequential processing.

Cascade R-CNN can be viewed as an ensemble of specialized classifiers operating at different quality levels. The final classification score is produced by the last stage, but each stage's classifier is optimized for the specific distribution of proposals it receives. This specialization yields significant improvements in mAP (mean Average Precision), particularly at higher IoU thresholds ([email protected]), where precise localization is critical for medical applications like lesion boundary delineation.

The core metric that defines Cascade R-CNN's stage-wise logic. IoU measures the overlap between a predicted bounding box and the ground truth annotation, calculated as the area of overlap divided by the area of union. Cascade R-CNN trains a sequence of detectors at increasing IoU thresholds (e.g., 0.5, 0.6, 0.7), where each stage's output serves as a refined input for the next. This directly addresses the overfitting and distribution mismatch problems that occur when a single detector is forced to handle proposals of widely varying quality.

The mathematical mechanism refined at each cascade stage. Bounding box regression learns a transformation function that maps a proposed region to a tighter, more accurate box around the object. In Cascade R-CNN, each regression head is specialized for a specific quality band of proposals. The first stage handles coarse localization, while subsequent stages perform fine-grained adjustments, progressively reducing the delta between the predicted and ground truth coordinates to achieve high-quality lesion localization.

The front-end component that feeds the cascade. The RPN is a fully convolutional network that scans the feature map and outputs a set of rectangular object proposals, each with an objectness score. In a Cascade R-CNN pipeline, the RPN generates the initial, relatively noisy proposals that are then sequentially refined by the cascade stages. The quality of these initial proposals directly impacts the efficiency of the entire multi-stage detection process.

The two-stage predecessor that Cascade R-CNN extends. Faster R-CNN introduced the RPN to enable near real-time detection by sharing convolutional features between the proposal and detection networks. Cascade R-CNN generalizes this architecture by replacing the single detection head with a sequence of detection heads. This addresses Faster R-CNN's fundamental limitation: a single regressor optimized for a fixed IoU threshold (typically 0.5) produces suboptimal boxes when asked to refine high-quality proposals at inference.

The post-processing step that eliminates redundant detections after the cascade. NMS sorts bounding boxes by confidence score and iteratively removes boxes that have a high IoU overlap with a higher-scoring box. Cascade R-CNN's output boxes are inherently higher quality and more precisely localized, which reduces the ambiguity during NMS and results in cleaner final detections with fewer false positives, particularly for clustered abnormalities in dense tissue.

A training strategy that synergizes with the cascade architecture. Hard negative mining identifies background regions that the model confidently but incorrectly classifies as objects (false positives) and explicitly re-trains on them. In Cascade R-CNN, each stage naturally encounters increasingly difficult negatives as the IoU threshold rises. The architecture's staged design implicitly performs a form of online hard negative mining, forcing subsequent detectors to focus on the most challenging background regions that survive earlier filtering.