Sparse Partial Least Squares (sPLS)

sPLS imposes an L1-norm constraint on the loading vectors during the deflation step of the NIPALS algorithm. This forces many feature weights to exactly zero, performing embedded feature selection simultaneously with dimension reduction. The sparsity parameter controls the number of selected variables per component.

• Each latent component is a sparse linear combination of the original features
• The penalty is applied via soft-thresholding in the iterative SVD step
• Unlike PCA, components are supervised—they maximize covariance with the response

Sparse Partial Least Squares Discriminant Analysis (sPLS-DA) adapts sPLS for categorical outcomes by encoding class labels as a dummy response matrix. It identifies the minimal set of features that best separate predefined groups, making it a workhorse for diagnostic biomarker panel discovery.

• Selects features that discriminate between disease vs. control, or multiple subtypes
• Outputs sparse loading vectors showing which metabolites, genes, or proteins drive class separation
• Often visualized via clustered image maps of the selected features

A key advantage of sPLS is the ability to specify a different number of selected features for each latent component. This allows the first component to capture broad, global variation with many features, while subsequent components isolate subtle, local effects with fewer variables.

• Parameter keepX controls the number of features retained per component
• Cross-validation on prediction error or classification accuracy guides tuning
• Prevents the model from overfitting to noise in high-dimensional spaces

Standard LASSO arbitrarily picks one representative from a group of highly correlated features. sPLS, inheriting the PLS framework, handles multicollinearity gracefully by projecting correlated variables onto the same latent direction. This is critical in genomics where genes operate in co-regulated modules.

• Correlated biomarkers can be selected together if they contribute to the same component
• Avoids the instability of LASSO in the presence of linkage disequilibrium or metabolic pathway correlations
• Produces more reproducible biomarker signatures across studies

Sparse variants like sGCCA (sparse Generalized Canonical Correlation Analysis) extend sPLS to simultaneously analyze multiple data blocks measured on the same samples—e.g., mRNA, miRNA, and proteomics. It identifies a common latent structure and selects the key features from each block that drive the correlation.

• Finds multi-omics biomarker panels that are jointly associated with an outcome
• Each block gets its own sparsity parameter
• Reveals cross-platform molecular relationships

The unsupervised dimensionality reduction technique that Partial Least Squares extends for supervised settings. PCA finds orthogonal linear combinations of features that sequentially maximize explained variance.

• Produces principal components ordered by eigenvalue magnitude
• The first component captures the direction of maximum variance in the data
• Unlike sPLS, PCA ignores the response variable entirely
• Components are dense: every original feature contributes to each component
• Sparse PCA addresses this interpretability gap by enforcing sparsity constraints on loadings

A robust framework that combines subsampling with high-dimensional selection algorithms to identify features consistently chosen across random data perturbations.

• Applies a selection algorithm (like LASSO) to many random subsamples of the data
• Computes the selection probability for each feature across all iterations
• Only retains features exceeding a predefined threshold of stability
• Provides rigorous false discovery rate control under mild assumptions
• Particularly valuable when sPLS or LASSO results vary significantly with small data changes

A variant of PCA that produces modified principal components with sparse loadings, directly addressing the interpretability challenge of dense PCA components.

• Formulated as an optimization problem with an L1 or elastic net penalty on loadings
• Each component uses only a subset of original features
• Trades some explained variance for dramatically improved interpretability
• The sparsity parameter controls the trade-off between variance captured and number of active features
• Shares the core philosophy of sPLS: meaningful latent structures should be sparse

The direct predecessor to sPLS. Standard PLS finds latent variables that maximize the covariance between predictors and response, but produces dense components where every original variable contributes.

• Constructs latent variables as linear combinations of original features
• Simultaneously decomposes both X (predictors) and Y (response) matrices
• Particularly effective for multicollinear, high-dimensional data
• The number of latent variables is a critical tuning parameter
• sPLS adds L1 penalization to the loading vectors, forcing many weights to zero and enabling simultaneous dimensionality reduction and feature selection