Test-retest reproducibility is the assessment of measurement variation in quantitative imaging features derived from repeated scans of the same subject with no intervening biological change. It isolates the inherent noise of the acquisition system—including scanner calibration drift, patient positioning, and physiological motion—from true pathological progression. A feature with high reproducibility yields near-identical values across short-interval scans, confirming its suitability as a robust biomarker.
Glossary
Test-Retest Reproducibility

What is Test-Retest Reproducibility?
Test-retest reproducibility quantifies the stability of radiomic feature measurements when imaging is repeated on the same subject under identical conditions within a short interval, ensuring observed biological changes are not confounded by scanner noise.
This metric is foundational to robust feature selection in radiomics pipelines. Features failing test-retest analysis are discarded before downstream modeling, as their variance is dominated by technical rather than biological factors. Reproducibility is typically quantified using the intraclass correlation coefficient (ICC) or concordance correlation coefficient (CCC), with features exceeding a threshold of 0.85 considered stable enough for clinical decision support and multi-center trial harmonization.
Key Factors Influencing Reproducibility
Test-retest reproducibility in radiomics is not a monolithic property but a composite metric influenced by a chain of technical and biological variables. The following factors determine whether a radiomic feature can be considered a stable, reliable biomarker or merely statistical noise.
Acquisition Parameter Standardization
The single most significant source of variability. Reproducibility is highly sensitive to deviations in the imaging protocol.
- Tube Current (mAs) and Voltage (kVp): Variations alter image noise texture, directly corrupting GLCM and GLRLM features.
- Slice Thickness: Thicker slices introduce partial volume averaging, destroying the fidelity of Shape Features and fine textures.
- Reconstruction Kernel: Sharp kernels (e.g., Bone) amplify noise, while smooth kernels blur edges. Switching kernels between scans can render First-Order Statistics incomparable.
- Pitch and Field of View: Alter the spatial resolution and voxel dimensions, requiring strict protocol lock-down for longitudinal studies.
Physiological Motion Artifacts
Biological noise sources that cannot be fully eliminated but must be assessed for their impact on feature stability.
- Respiratory Motion: In thoracic and abdominal imaging, inconsistent breath-hold depths displace Volume of Interest (VOI) boundaries, introducing non-biological texture variance.
- Cardiac Pulsation: Transmits pulsatile motion to adjacent structures (e.g., lung bases, liver dome), degrading NGTDM features that rely on local neighborhood intensity differences.
- Peristalsis: Uncontrolled bowel motion changes the shape and density distribution of abdominal organs between scans, severely impacting GLSZM reproducibility.
- Mitigation: Features robust against these artifacts are classified as Robust Features and prioritized during model building.
Pre-Processing Pipeline Consistency
Computational reproducibility requires deterministic, locked-down image preparation steps before feature extraction.
- Voxel Resampling: Interpolation to isotropic voxels (e.g., 1x1x1 mm³) must use identical algorithms (linear vs. B-spline) to avoid introducing systematic spatial biases.
- Intensity Discretization: The number of discrete bins (e.g., 32, 64) fundamentally alters texture matrix probabilities. A fixed bin width strategy is recommended by the Image Biomarker Standardisation Initiative (IBSI) to ensure cross-scanner stability.
- HU Rescaling: For CT, absolute Hounsfield Unit calibration is critical. Features derived from absolute intensity values are meaningless if the scanner's water attenuation baseline drifts.
- Normalization: Applying z-score normalization or histogram matching can stabilize features but may mask genuine biological changes if over-applied.
Region of Interest (ROI) Delineation Variability
The 'jagged edge' problem. Manual or semi-automated segmentation introduces inter- and intra-observer variability that propagates into feature values.
- Boundary Uncertainty: Tumors with infiltrative margins (e.g., glioblastoma) exhibit poor segmentation agreement, directly destabilizing Shape Features like sphericity and surface area.
- Margin Inclusion: Including or excluding peritumoral tissue alters the texture context, switching the biological meaning of the GLCM correlation feature.
- Automated Segmentation: Deep learning-based auto-contouring (e.g., nnU-Net) significantly improves reproducibility over manual tracing, but the specific model version must be locked as a variable.
- Test-Retest Metric: The Dice Similarity Coefficient between two segmentations is a prerequisite metric; features from volumes with Dice < 0.8 are typically excluded from analysis.
Feature Family Intrinsic Stability
Not all feature classes are created equal. Empirical test-retest studies consistently rank feature families by their inherent robustness to noise.
- High Stability: First-Order Statistics (mean, median) and basic Shape Features (volume, maximum 3D diameter) show excellent Concordance Correlation Coefficients (CCC > 0.9).
- Moderate Stability: GLCM features like homogeneity and contrast are generally stable if discretization is fixed.
- Low Stability: GLRLM and GLSZM features, particularly those capturing high-frequency heterogeneity (e.g., Short Run Emphasis), are highly sensitive to noise and reconstruction kernel changes.
- Selection Strategy: A Robust Feature Selection filter, retaining only features with CCC > 0.85 on a test-retest phantom or patient cohort, is mandatory before building a Radiomic Signature.
Scanner Hardware Drift and Calibration
Long-term reproducibility requires monitoring the physical state of the imaging hardware, not just the software.
- Detector Aging: Gradual degradation of CT detector elements introduces ring artifacts and non-uniform noise profiles that corrupt Wavelet Transform sub-bands.
- Magnet Shimming (MRI): B0 field inhomogeneity drifts over time, distorting geometry and intensity uniformity, which invalidates Shape Features and Entropy measurements.
- Quality Assurance Phantoms: Routine scanning of standardized phantoms (e.g., ACR CT phantom) allows for Feature Harmonization by mapping feature values to a stable physical reference.
- ComBat Harmonization: A statistical technique originally from genomics, ComBat Harmonization can retrospectively correct for known scanner batch effects, preserving biological variance while removing technical drift.
Frequently Asked Questions
Addressing common questions about the assessment and clinical importance of measurement stability in quantitative imaging biomarkers.
Test-retest reproducibility is the quantitative assessment of the stability of radiomic feature measurements when the same subject is imaged twice within a short time interval under identical acquisition and reconstruction parameters. The core principle is that if no biological change has occurred, any variation in the extracted feature value is attributable to non-biological noise—such as scanner thermal drift, patient micro-motion, or stochastic reconstruction artifacts. This is formally quantified using statistical metrics like the Concordance Correlation Coefficient (CCC) and Intraclass Correlation Coefficient (ICC). A feature with high test-retest reproducibility is considered a reliable candidate for a robust imaging biomarker, whereas features that fluctuate wildly between scans are typically discarded during the robust feature selection phase to prevent spurious clinical associations.
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.

Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.

Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
Related Terms
Understanding test-retest reproducibility requires familiarity with the preprocessing standards and statistical methods that directly influence feature stability.
ComBat Harmonization
A statistical batch-effect correction method originally developed for genomics and adapted to radiomics to standardize feature values across multiple imaging centers. ComBat uses an empirical Bayes framework to:
- Estimate and remove additive and multiplicative scanner effects
- Preserve biological variability associated with the outcome of interest
- Harmonize features without requiring phantom calibration data When test-retest variability is dominated by multi-center scanner differences rather than true biological change, ComBat harmonization is essential for restoring reproducibility.
Intensity Discretization
The process of converting continuous image intensity values into a finite number of discrete bins, a critical pre-processing step for texture matrix calculation. Key parameters include:
- Fixed bin count: Divides the intensity range into a set number of bins (e.g., 32, 64, 128)
- Fixed bin width: Sets a constant bin size in Hounsfield Units or original units Discretization choices dramatically impact test-retest reproducibility. Fixed bin width methods generally yield more stable texture features across repeat scans than fixed bin count approaches.
Voxel Resampling
The process of interpolating medical image data to create isotropic voxels, ensuring spatial measurements are consistent across all three dimensions. Common resampling targets include:
- 1×1×1 mm³ for high-resolution analysis
- 2×2×2 mm³ for computational efficiency
- In-plane resolution matching to preserve native axial detail Non-isotropic voxels cause rotationally variant texture features. Resampling to isotropic dimensions is a prerequisite for achieving directionally stable GLCM and GLRLM features across repeat imaging sessions.
Robust Feature Selection
A dimensionality reduction strategy that identifies and retains only radiomic features demonstrating high stability against test-retest and inter-observer variability. Common stability metrics include:
- Concordance Correlation Coefficient (CCC): Measures agreement between repeat measurements; features with CCC > 0.85 are typically retained
- Intraclass Correlation Coefficient (ICC): Quantifies reliability across multiple raters or timepoints
- Coefficient of Variation (CV): Expresses within-subject variability as a percentage of the mean This filtering step eliminates non-reproducible features before model training, preventing overfitting to measurement noise.

About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us