Sure Independence Screening (SIS)

The defining theoretical guarantee of SIS is the sure screening property. Under certain regularity conditions, as the sample size n approaches infinity, the probability that the screened subset contains all truly relevant features approaches one. This ensures that no important variables are lost during the aggressive dimensionality reduction stage. The property relies on the assumption that the marginal signal of true features does not decay too quickly, allowing them to be ranked highly by simple marginal correlation measures even in the presence of a massive number of noise variables.

Standard SIS can fail when a feature is marginally uncorrelated with the response but jointly important with other features, or when irrelevant features exhibit high marginal correlation due to spurious collinearity with true predictors. Iterative SIS (ISIS) addresses these limitations by alternating between screening and selection:

• Step 1: Apply SIS to select a moderate set of features.
• Step 2: Fit a refined model (e.g., LASSO or SCAD) on the selected set.
• Step 3: Compute residuals from the fitted model.
• Step 4: Screen the remaining features using their marginal correlation with the residuals. This iterative process captures features that have a significant conditional contribution but weak marginal signal, substantially improving screening accuracy in complex correlation structures.

The SIS framework extends beyond the standard linear model to generalized linear models (GLMs) for non-Gaussian responses. For binary classification, the marginal utility measure is replaced by the maximum marginal likelihood estimator of each feature in a univariate logistic regression. For count data, a Poisson regression framework is used. The key principle remains: each feature is evaluated independently using a model from the appropriate exponential family, and the top d features with the largest marginal deviance reduction or absolute coefficient estimate are retained. This allows SIS to serve as a pre-filter for ultra-high-dimensional classification and survival analysis problems common in genomics and medical imaging.

Standard SIS relies on Pearson correlation, which captures only linear relationships and can miss features with strong non-linear dependencies on the response. Distance Correlation SIS (DC-SIS) replaces the Pearson coefficient with the distance correlation measure, a non-parametric statistic that equals zero if and only if two random vectors are statistically independent. This model-free approach ensures the sure screening property holds for arbitrary functional relationships without requiring a correctly specified model. DC-SIS is particularly valuable in exploratory biomarker discovery where the functional form linking a molecular feature to a disease outcome is unknown and potentially highly non-linear.

A critical practical challenge in SIS is selecting the sub-model size d—the number of features to retain after screening. Choosing d too small risks missing true signals; choosing d too large burdens the second-stage selection method. Knockoff SIS integrates the model-X knockoff framework to provide finite-sample false discovery rate (FDR) control. Synthetic knockoff variables that mimic the correlation structure of the original features are generated and act as negative controls. Features are selected only if their marginal utility significantly exceeds that of their knockoff counterpart, providing a data-adaptive threshold for d with explicit FDR guarantees, a crucial property for reproducible biomarker validation.

In many biological applications, features exhibit natural groupings (e.g., genes in the same pathway or SNPs in the same gene). Group SIS screens entire groups of features simultaneously by evaluating the aggregate marginal utility of a group, often using a group-wise composite score or the norm of the group's coefficient vector from a multivariate marginal model. For problems with multiple correlated responses (e.g., predicting several related clinical outcomes), Multivariate SIS screens features based on their joint marginal relationship with the entire response matrix, using measures like the trace of the covariance matrix or canonical correlation. These extensions preserve the computational scalability of SIS while respecting the structural dependencies inherent in multi-omics data.