nnU-Net

Before any training begins, nnU-Net analyzes the target dataset to extract a unique data fingerprint. This includes image modality (CT, MRI), voxel spacing, intensity distribution, and class imbalance ratios. These dataset properties—not human heuristics—drive all subsequent configuration decisions, ensuring the pipeline is genuinely adapted to the specific clinical acquisition protocol.

nnU-Net applies a fixed set of heuristic rules to configure preprocessing based on the data fingerprint:

• Resampling: All images are resampled to the median voxel spacing of the dataset to standardize resolution.
• Intensity Normalization: CT scans receive global percentile clipping and z-score normalization; MRI scans receive z-score normalization per image to handle bias field variability.
• Cropping: Non-zero regions are cropped to reduce computational overhead while preserving all foreground anatomy.

The framework dynamically generates a 3D U-Net architecture tailored to the dataset's geometry. Key parameters derived automatically include:

• Patch size: Selected to cover the median image shape while fitting within GPU memory constraints (typically 12GB).
• Pooling operations: Determined by the patch size to ensure feature maps can be downsampled to a 4×4×4 bottleneck.
• Batch size: Maximized to fill available GPU memory.
• Encoder-decoder depth: Adjusted so that the receptive field covers the full patch.

When the required patch size cannot capture sufficient anatomical context—common in whole-body or multi-organ segmentation—nnU-Net automatically triggers a two-stage cascade. Stage one operates on heavily downsampled images to learn coarse anatomy. Stage two refines the segmentation at full resolution using the stage one predictions as an additional input channel, effectively providing spatial prior information.

Unlike most frameworks that ignore post-processing, nnU-Net automatically evaluates whether connected component analysis improves validation performance. For each class, it tests removing all but the largest connected component. If this step increases the Dice Similarity Coefficient on the validation set, it is retained; otherwise, it is discarded. This simple rule eliminates false-positive islands in organ segmentation.

nnU-Net trains five separate models using 5-fold cross-validation by default. At inference, predictions from all five folds are averaged to produce the final segmentation. This ensemble strategy consistently yields a 1-2% improvement in Dice score over single-model predictions and provides built-in uncertainty estimation by measuring inter-fold variance.

The raw input format nnU-Net ingests. A DICOM Series is a sequence of 2D slices acquired in a single scan, sharing identical modality, orientation, and temporal parameters. nnU-Net's data fingerprinting automatically parses DICOM headers to extract voxel spacing, slice thickness, and modality—critical metadata that drives its automated preprocessing decisions without manual configuration.

The ground truth and output format. A segmentation mask is a voxel-wise label map classifying each 3D coordinate into anatomical structures or lesions. nnU-Net expects training masks as integer-valued volumes where 0 is background and positive integers denote distinct classes. Its loss function computes the Dice Similarity Coefficient directly on these masks, and its post-processing automatically refines mask boundaries using connected component analysis.

The primary optimization target. The Dice Similarity Coefficient measures spatial overlap between predicted and ground truth segmentation masks, ranging from 0 (no overlap) to 1 (perfect agreement). nnU-Net uses a soft Dice loss during training and reports DSC as its main validation metric. The framework's self-configuration selects architectures and patch sizes specifically to maximize this coefficient per dataset.

The foundational architecture nnU-Net builds upon. The 3D U-Net is a volumetric convolutional neural network with an encoder-decoder structure and skip connections that propagate fine spatial detail across resolutions. nnU-Net automatically configures the U-Net's depth, number of pooling operations, kernel sizes, and instance normalization based on the dataset's image geometry and GPU memory constraints, eliminating architectural guesswork.

A critical MRI preprocessing step automated by nnU-Net. Bias field correction removes low-frequency intensity non-uniformity artifacts caused by magnetic field inhomogeneities in MR images. nnU-Net's rule-based expert system detects MRI modalities and automatically applies the N4ITK algorithm to normalize intensities before training, ensuring the network learns anatomical features rather than scanner artifacts.

The ecosystem nnU-Net operates within. MONAI (Medical Open Network for AI) is a PyTorch-based framework providing domain-optimized components for medical imaging deep learning. While nnU-Net is a standalone self-configuring pipeline, its principles—data fingerprinting, automated augmentation, and standardized preprocessing—have influenced MONAI's design philosophy for reproducible research and clinical deployment workflows.