Intraclass Correlation Coefficient (ICC)

Used when each subject is rated by a different set of k raters randomly selected from a larger population. This model treats rater effects as random rather than fixed.

• Application: Assessing reliability of fully automated radiomic feature extraction where the 'rater' is a single measurement instance.
• Variance Components: Partitions total variance into between-subject and within-subject (error) variance only.
• Key Distinction: Does not separate systematic rater bias from residual error; all within-subject variation is pooled.
• Limitation: Underestimates reliability if systematic differences between specific scanners or raters exist.

Applies when the same set of k raters is drawn from a larger population and rates all subjects. Both subject and rater effects are modeled as random variables.

• Application: Test-retest studies where the same set of scanners or radiologists evaluates all patients, and you want to generalize to other scanners.
• Variance Components: Decomposes variance into subject, rater, and residual error components.
• Generalizability: Results can be generalized to other raters from the same population, making it ideal for multi-center radiomic harmonization studies.
• Assumption: Raters are exchangeable samples from a defined population of potential raters.

Used when the k raters are the only raters of interest—they are considered fixed effects, not a random sample. Inference is restricted to these specific raters.

• Application: Evaluating reliability between two specific MRI scanners or two specific radiologists at a single institution.
• Variance Components: Subject effects are random; rater effects are fixed. Only subject and residual variance are estimated.
• Consistency vs. Agreement: The consistency form (ICC(3,1)) excludes systematic rater bias from the denominator, measuring correlation. The absolute agreement form includes it.
• Critical for Radiomics: Use absolute agreement when feature values must be numerically identical across scanners; use consistency when rank-ordering of patients suffices.

Estimates the reliability of the mean of k measurements rather than a single measurement. Always higher than single-measure ICCs.

• Formula Relationship: ICC(k) = k * ICC(1) / [1 + (k-1) * ICC(1)], derived from the Spearman-Brown prophecy formula.
• Application: When radiomic features are averaged across multiple acquisitions or multiple segmentations before being used in a predictive model.
• Reporting Standard: Always specify whether reporting single or average measures. The IBSI guidelines recommend reporting both.
• Practical Impact: Averaging 3 segmentations can substantially boost reliability for features with moderate single-measure ICC.

A critical distinction within two-way models that determines whether systematic bias between raters is penalized in the reliability estimate.

• Absolute Agreement: Measures whether raters assign the exact same value. Systematic additive or multiplicative bias reduces the ICC. Essential for radiomic feature harmonization.
• Consistency: Measures whether raters rank subjects in the same order, ignoring systematic offsets. A feature can have high consistency but low agreement if one scanner consistently reads higher.
• Radiomics Context: Use absolute agreement when building models that will be deployed across scanners. Use consistency when evaluating a feature's discriminatory power within a single scanner.
• IBSI Recommendation: The Image Biomarker Standardisation Initiative mandates reporting absolute agreement for multi-center reproducibility.

While standard ICC assumes continuous, normally distributed data, extensions exist for non-continuous radiomic feature types.

• Continuous ICC: The classic formulation using ANOVA mean squares. Requires normality and homoscedasticity; log-transformation may be needed for skewed radiomic features.
• Binary/Ordinal ICC: Uses a latent variable approach or generalized linear mixed models (GLMM) with a probit or logit link function for categorical radiomic features.
• Non-Parametric Alternatives: When normality assumptions are violated, consider the Concordance Correlation Coefficient (CCC) or Kendall's W as robust alternatives.
• Practical Guidance: First-order features like entropy often satisfy normality; texture features from GLCM or GLRLM frequently require transformation or non-parametric methods.

A specific application of ICC measuring the stability of radiomic features when the same subject is scanned twice within a short interval with no biological change. This isolates variance from scanner noise, patient positioning, and breath-hold inconsistencies. Features with low test-retest ICC are typically excluded from predictive models as they capture noise rather than true tumor phenotype.

ICC calculated across multiple radiologists or algorithms delineating the same Region of Interest (ROI). This quantifies how robust a feature is to boundary uncertainty. Shape features often show excellent inter-observer ICC, while texture features near tumor margins may degrade. Studies typically report this alongside the Dice Similarity Coefficient for the segmentation itself.

The process of binning continuous voxel intensity values into a finite number of discrete gray levels, a critical preprocessing step for texture matrix calculation. The choice of bin width directly impacts the reproducibility of second-order features. Standardizing discretization (e.g., fixed bin number vs. fixed bin width) is mandatory before comparing ICC values across studies, as inconsistent binning introduces artificial variance.

A dimensionality reduction strategy where only radiomic features exceeding a pre-defined ICC threshold (e.g., ICC > 0.8) are retained for downstream predictive modeling. This acts as a filter method, removing noisy features before applying algorithms like LASSO or mRMR. This approach ensures that the final radiomic signature is built on a foundation of robust, reproducible measurements.