Principal Component Analysis (PCA)

A variant of PCA that produces modified principal components with sparse loadings, meaning each component is a linear combination of only a small subset of the original features. This addresses PCA's primary limitation: interpretability.

• Uses L1 (LASSO) or elastic net penalties on component loadings
• Drives many feature weights to exactly zero
• Critical in genomics where you need to identify which specific genes drive a component
• Trade-off: explains slightly less variance than standard PCA but yields human-interpretable components

A group of algorithms that factorize a high-dimensional matrix V into two lower-dimensional, non-negative matrices W and H, such that V ≈ WH. Unlike PCA, NMF enforces a parts-based, additive representation.

• All elements in W and H are constrained to be ≥ 0
• Produces sparse, localized features rather than holistic components
• Widely used in digital pathology for extracting meaningful tissue patterns
• No orthogonality constraint, allowing overlapping representations
• Particularly effective when data is inherently non-negative (image pixels, word counts)

An unsupervised neural network architecture that learns a compressed, lower-dimensional representation by training the network to reconstruct its input from a bottleneck layer. Autoencoders capture non-linear relationships that PCA's linear transformation cannot.

• Encoder compresses input → bottleneck (latent representation) → decoder reconstructs
• Undercomplete autoencoders with bottleneck smaller than input dimension perform dimensionality reduction
• Denoising autoencoders learn robust features by reconstructing clean inputs from corrupted versions
• Variational autoencoders (VAEs) learn a probabilistic latent space for generative modeling
• Key advantage over PCA: can learn complex manifolds in the data

A dimensionality reduction method that integrates the feature selection capability of LASSO with the latent variable modeling of Partial Least Squares. Unlike PCA, which maximizes variance in the predictor space alone, sPLS finds sparse linear combinations that maximize covariance with a response variable.

• Supervised alternative to PCA for regression and classification problems
• Produces sparse weight vectors where irrelevant features receive zero weight
• Essential in biomarker discovery where you need both dimensionality reduction and feature selection
• Handles the n << p problem (more features than samples) common in genomics
• Implemented in the mixOmics R package for multi-omics integration

A deep learning architecture for unsupervised feature selection that uses a concrete relaxation of the discrete distribution in its bottleneck layer. Unlike standard autoencoders that learn dense representations, the concrete autoencoder learns to select a subset of the most informative input features.

• Uses the Gumbel-softmax (concrete) distribution to approximate discrete selection during training
• Anneals temperature parameter to converge toward hard feature selection
• Bridges the gap between embedded feature selection and deep representation learning
• Outperforms LASSO and standard autoencoders on high-dimensional biological datasets
• Provides a differentiable end-to-end feature selection pipeline

A systems biology method that finds clusters (modules) of highly correlated genes, summarizes each module by its first principal component (the module eigengene), and relates these modules to external clinical traits. WGCNA leverages PCA at the module level for biomarker discovery.

• Constructs a scale-free network from gene expression correlations
• Uses hierarchical clustering with dynamic tree cutting to identify modules
• Module eigengene = PC1 of the module's expression matrix
• Relates modules to traits via correlation between eigengenes and clinical variables
• Identifies hub genes within clinically significant modules as potential biomarkers