Inferensys

Glossary

Test-Retest Reproducibility

The assessment of the stability of radiomic feature measurements when imaging is repeated on the same subject under identical conditions within a short interval.
Data engineer managing feature store on laptop, feature definitions visible, casual data engineering session.
MEASUREMENT STABILITY

What is Test-Retest Reproducibility?

Test-retest reproducibility quantifies the stability of radiomic feature measurements when imaging is repeated on the same subject under identical conditions within a short interval, ensuring observed biological changes are not confounded by scanner noise.

Test-retest reproducibility is the assessment of measurement variation in quantitative imaging features derived from repeated scans of the same subject with no intervening biological change. It isolates the inherent noise of the acquisition system—including scanner calibration drift, patient positioning, and physiological motion—from true pathological progression. A feature with high reproducibility yields near-identical values across short-interval scans, confirming its suitability as a robust biomarker.

This metric is foundational to robust feature selection in radiomics pipelines. Features failing test-retest analysis are discarded before downstream modeling, as their variance is dominated by technical rather than biological factors. Reproducibility is typically quantified using the intraclass correlation coefficient (ICC) or concordance correlation coefficient (CCC), with features exceeding a threshold of 0.85 considered stable enough for clinical decision support and multi-center trial harmonization.

STABILITY DETERMINANTS

Key Factors Influencing Reproducibility

Test-retest reproducibility in radiomics is not a monolithic property but a composite metric influenced by a chain of technical and biological variables. The following factors determine whether a radiomic feature can be considered a stable, reliable biomarker or merely statistical noise.

01

Acquisition Parameter Standardization

The single most significant source of variability. Reproducibility is highly sensitive to deviations in the imaging protocol.

  • Tube Current (mAs) and Voltage (kVp): Variations alter image noise texture, directly corrupting GLCM and GLRLM features.
  • Slice Thickness: Thicker slices introduce partial volume averaging, destroying the fidelity of Shape Features and fine textures.
  • Reconstruction Kernel: Sharp kernels (e.g., Bone) amplify noise, while smooth kernels blur edges. Switching kernels between scans can render First-Order Statistics incomparable.
  • Pitch and Field of View: Alter the spatial resolution and voxel dimensions, requiring strict protocol lock-down for longitudinal studies.
02

Physiological Motion Artifacts

Biological noise sources that cannot be fully eliminated but must be assessed for their impact on feature stability.

  • Respiratory Motion: In thoracic and abdominal imaging, inconsistent breath-hold depths displace Volume of Interest (VOI) boundaries, introducing non-biological texture variance.
  • Cardiac Pulsation: Transmits pulsatile motion to adjacent structures (e.g., lung bases, liver dome), degrading NGTDM features that rely on local neighborhood intensity differences.
  • Peristalsis: Uncontrolled bowel motion changes the shape and density distribution of abdominal organs between scans, severely impacting GLSZM reproducibility.
  • Mitigation: Features robust against these artifacts are classified as Robust Features and prioritized during model building.
03

Pre-Processing Pipeline Consistency

Computational reproducibility requires deterministic, locked-down image preparation steps before feature extraction.

  • Voxel Resampling: Interpolation to isotropic voxels (e.g., 1x1x1 mm³) must use identical algorithms (linear vs. B-spline) to avoid introducing systematic spatial biases.
  • Intensity Discretization: The number of discrete bins (e.g., 32, 64) fundamentally alters texture matrix probabilities. A fixed bin width strategy is recommended by the Image Biomarker Standardisation Initiative (IBSI) to ensure cross-scanner stability.
  • HU Rescaling: For CT, absolute Hounsfield Unit calibration is critical. Features derived from absolute intensity values are meaningless if the scanner's water attenuation baseline drifts.
  • Normalization: Applying z-score normalization or histogram matching can stabilize features but may mask genuine biological changes if over-applied.
04

Region of Interest (ROI) Delineation Variability

The 'jagged edge' problem. Manual or semi-automated segmentation introduces inter- and intra-observer variability that propagates into feature values.

  • Boundary Uncertainty: Tumors with infiltrative margins (e.g., glioblastoma) exhibit poor segmentation agreement, directly destabilizing Shape Features like sphericity and surface area.
  • Margin Inclusion: Including or excluding peritumoral tissue alters the texture context, switching the biological meaning of the GLCM correlation feature.
  • Automated Segmentation: Deep learning-based auto-contouring (e.g., nnU-Net) significantly improves reproducibility over manual tracing, but the specific model version must be locked as a variable.
  • Test-Retest Metric: The Dice Similarity Coefficient between two segmentations is a prerequisite metric; features from volumes with Dice < 0.8 are typically excluded from analysis.
05

Feature Family Intrinsic Stability

Not all feature classes are created equal. Empirical test-retest studies consistently rank feature families by their inherent robustness to noise.

  • High Stability: First-Order Statistics (mean, median) and basic Shape Features (volume, maximum 3D diameter) show excellent Concordance Correlation Coefficients (CCC > 0.9).
  • Moderate Stability: GLCM features like homogeneity and contrast are generally stable if discretization is fixed.
  • Low Stability: GLRLM and GLSZM features, particularly those capturing high-frequency heterogeneity (e.g., Short Run Emphasis), are highly sensitive to noise and reconstruction kernel changes.
  • Selection Strategy: A Robust Feature Selection filter, retaining only features with CCC > 0.85 on a test-retest phantom or patient cohort, is mandatory before building a Radiomic Signature.
06

Scanner Hardware Drift and Calibration

Long-term reproducibility requires monitoring the physical state of the imaging hardware, not just the software.

  • Detector Aging: Gradual degradation of CT detector elements introduces ring artifacts and non-uniform noise profiles that corrupt Wavelet Transform sub-bands.
  • Magnet Shimming (MRI): B0 field inhomogeneity drifts over time, distorting geometry and intensity uniformity, which invalidates Shape Features and Entropy measurements.
  • Quality Assurance Phantoms: Routine scanning of standardized phantoms (e.g., ACR CT phantom) allows for Feature Harmonization by mapping feature values to a stable physical reference.
  • ComBat Harmonization: A statistical technique originally from genomics, ComBat Harmonization can retrospectively correct for known scanner batch effects, preserving biological variance while removing technical drift.
TEST-RETEST RELIABILITY

Frequently Asked Questions

Addressing common questions about the assessment and clinical importance of measurement stability in quantitative imaging biomarkers.

Test-retest reproducibility is the quantitative assessment of the stability of radiomic feature measurements when the same subject is imaged twice within a short time interval under identical acquisition and reconstruction parameters. The core principle is that if no biological change has occurred, any variation in the extracted feature value is attributable to non-biological noise—such as scanner thermal drift, patient micro-motion, or stochastic reconstruction artifacts. This is formally quantified using statistical metrics like the Concordance Correlation Coefficient (CCC) and Intraclass Correlation Coefficient (ICC). A feature with high test-retest reproducibility is considered a reliable candidate for a robust imaging biomarker, whereas features that fluctuate wildly between scans are typically discarded during the robust feature selection phase to prevent spurious clinical associations.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.