nnU-Net (no-new-Net)

Before any training begins, nnU-Net extracts a dataset fingerprint—a statistical summary of the input data. This includes voxel spacing distributions, intensity value ranges, and class imbalance ratios. The fingerprint captures the physical resolution and contrast characteristics unique to each modality, such as the anisotropic spacing typical of CT scans versus the near-isotropic acquisitions in high-resolution MRI. This automated analysis replaces the manual inspection and heuristic tuning that engineers traditionally perform when adapting a model to a new imaging protocol.

nnU-Net translates the dataset fingerprint into a fully specified training pipeline using a set of heuristic rules derived from extensive empirical testing. These rules govern:

• Preprocessing: Resampling strategy (anisotropic vs. isotropic), intensity normalization method (z-score vs. CT windowing).
• Network Topology: Input patch size, number of pooling operations, kernel sizes, and batch size, all adapted to the median voxel spacing and GPU memory constraints.
• Training Dynamics: Loss function selection (compound Dice + Cross-Entropy), learning rate schedules, and data augmentation policies. This rule-based approach ensures that design choices are transparent, reproducible, and grounded in measurable data properties rather than arbitrary preference.

For datasets where the required spatial context exceeds what a single 3D U-Net can process within GPU memory limits, nnU-Net automatically activates a two-stage cascade. The first stage operates on heavily downsampled images to coarsely localize the target region. The second stage refines the segmentation at full resolution, but only within the bounding box predicted by the first model. This cascade strategy is critical for tasks like whole-body organ segmentation or liver tumor delineation, where the organ of interest occupies a small fraction of the total scan volume but must be understood within its broader anatomical context.

Instead of relying on a single model, nnU-Net trains a diverse ensemble of five models using cross-validation folds and automatically determines the optimal combination. The framework evaluates all possible subsets of the 5-fold ensemble (2^5 - 1 = 31 combinations) on a held-out validation set and selects the subset that maximizes the target metric, typically Dice score. This exhaustive search is computationally feasible because inference is fast relative to training. The selected ensemble is then used for final prediction, consistently outperforming any single model by averaging out fold-specific biases and capturing complementary error patterns.

nnU-Net automatically searches for and applies instance-level post-processing rules that improve segmentation coherence. The framework analyzes connected components in the predicted masks and evaluates whether enforcing or removing specific components—based on their size relative to the median object size in the training set—improves validation performance. For example, in organ segmentation, this step eliminates small, spurious false-positive blobs outside the plausible anatomical region. The decision to apply these rules is made purely on quantitative grounds, avoiding manual cleanup scripts that introduce human bias.

The nnU-Net framework makes no assumptions about the imaging modality, anatomical region, or segmentation task. It has been validated across 23 international biomedical segmentation challenges spanning CT, MRI, PET, ultrasound, and microscopy data. The same codebase, without any modification, configures itself for tasks as diverse as brain tumor sub-region segmentation in multi-parametric MRI, lung nodule detection in low-dose CT, and cell membrane boundary delineation in electron microscopy. This universality demonstrates that careful, data-driven pipeline configuration can match or exceed the performance of highly specialized, hand-crafted solutions.

The TotalSegmentator model, built on nnU-Net, segments 104 anatomical structures in whole-body CT scans in under 2 minutes. This enables automated organ volumetry, surgical planning, and opportunistic screening by extracting quantitative biomarkers from routine imaging.

• Processes full-body CT volumes with isotropic resampling
• Handles anatomical variants without manual intervention
• Deployed in radiology workflows for trauma assessment and pre-operative planning

nnU-Net won the BraTS (Brain Tumor Segmentation) challenge by automatically segmenting enhancing tumor, peritumoral edema, and necrotic core from multi-parametric MRI. The framework's 5-fold cross-validation strategy and test-time augmentation deliver robust performance across heterogeneous acquisition protocols.

• Segments T1, T1ce, T2, and FLAIR sequences simultaneously
• Handles multi-class segmentation with severe class imbalance
• Used in clinical trials for volumetric response assessment

Radiation oncology departments use nnU-Net to delineate Organs-at-Risk (OARs) and Gross Tumor Volumes (GTVs) from planning CTs. The framework's ability to handle anisotropic voxel spacing and produce smooth 3D surfaces makes it suitable for generating DICOM RT Structure Sets directly.

• Segments 20+ OARs for head-and-neck radiotherapy
• Integrates with treatment planning systems via DICOM export
• Reduces inter-observer contouring variability by 30-40%

nnU-Net segments the left and right ventricles, atria, and myocardium from cardiac cine MRI across all phases of the cardiac cycle. This enables automated ejection fraction calculation, wall motion analysis, and strain quantification without manual contouring.

• Handles 2D+t and 3D+t data with temporal consistency
• Processes short-axis and long-axis views simultaneously
• Achieves Dice scores exceeding 0.92 on multi-center datasets

The framework automatically segments liver, spleen, kidneys, pancreas, and abdominal vessels from T1-weighted and T2-weighted MRI sequences. nnU-Net's built-in intensity normalization and bias field correction handle the inherent intensity inhomogeneities of MRI.

• Adapts preprocessing pipeline per modality automatically
• Handles respiratory motion artifacts through robust augmentation
• Used for volumetric liver assessment and transplant planning

While nnU-Net was designed for radiology, its self-configuring principles have been adapted for gigapixel pathology image segmentation. The framework tiles WSIs into manageable patches, segments tissue regions, and reconstructs the full-slide segmentation map using connected component analysis.

• Identifies tumor regions, stroma, and necrosis in H&E slides
• Handles staining variability through adaptive color normalization
• Integrates with digital pathology platforms for computational pathology workflows