Dimensionality Reduction

A linear dimensionality reduction technique that transforms data into a set of orthogonal principal components ordered by the variance they explain. PCA identifies the directions (eigenvectors) of maximum variance in the feature space.

• Mechanism: Eigendecomposition of the covariance matrix or Singular Value Decomposition (SVD).
• Output: A ranked set of uncorrelated variables; the first few components often capture the dominant biological signal.
• Clinical Use Case: Reducing thousands of gene expression values to 2-3 components for visualizing broad tumor vs. normal tissue separation.
• Limitation: Assumes linear relationships; cannot capture complex, curved manifolds in single-cell data.

A non-linear dimensionality reduction algorithm optimized for visualizing high-dimensional data in 2D or 3D. t-SNE converts high-dimensional Euclidean distances into conditional probabilities representing similarities.

• Key Parameter: Perplexity—a smooth measure of the effective number of neighbors, typically set between 5 and 50.
• Crowding Problem Solved: Uses a heavy-tailed Student-t distribution in the low-dimensional space to alleviate the crowding problem, allowing moderately distant points to be placed further apart.
• Clinical Use Case: Visualizing immune cell subsets from mass cytometry data, where distinct populations form clearly separated islands.
• Limitation: Primarily preserves local structure; global distances between clusters are not interpretable. Non-deterministic output.

A manifold learning technique that balances local and global structure preservation better than t-SNE. UMAP constructs a fuzzy topological representation of the high-dimensional data using k-nearest neighbor graphs.

• Mathematical Foundation: Based on Riemannian geometry and algebraic topology; assumes data is uniformly distributed on a locally connected manifold.
• Key Parameters: n_neighbors (controls local vs. global focus) and min_dist (controls point packing tightness).
• Clinical Use Case: Visualizing patient trajectories in longitudinal studies, where the continuous nature of disease progression is preserved as a connected path.
• Advantage: Significantly faster than t-SNE on large datasets and preserves more of the global data structure, making cluster distances somewhat meaningful.

A neural network architecture trained to reconstruct its input through a bottleneck layer, forcing the network to learn a compressed, latent representation of the data.

• Architecture: An encoder compresses the input into a low-dimensional code; a decoder reconstructs the original input from that code.
• Variational Autoencoders (VAEs): Enforce a probabilistic distribution on the latent space, enabling generative capabilities and smooth interpolation between patient profiles.
• Clinical Use Case: Learning a compressed representation of electronic health records to identify latent disease phenotypes that are not captured by discrete billing codes.
• Advantage: Can learn highly complex, non-linear manifolds that PCA cannot capture, and can be adapted to specific data types (e.g., convolutional for images).

A classical technique that finds a low-dimensional embedding of data points while preserving their pairwise distances as faithfully as possible.

• Metric vs. Non-Metric: Metric MDS preserves actual distances; Non-Metric MDS preserves only the rank order of distances.
• Stress Function: Minimizes a loss function (stress) that measures the discrepancy between high-dimensional and low-dimensional distances.
• Clinical Use Case: Visualizing the genetic distance between different tumor samples to confirm that samples from the same tissue of origin cluster together.
• Relationship to PCA: Classical MDS with Euclidean distances is equivalent to PCA, providing a direct link between linear and distance-based methods.

A critical distinction in dimensionality reduction: feature selection retains a subset of original variables, while feature extraction creates new composite variables.

• Feature Selection: Methods like LASSO regularization or mutual information scores. Retains interpretability because original biomarkers (e.g., specific genes) are kept.
• Feature Extraction: Methods like PCA or autoencoders. Creates new latent variables that are combinations of originals, maximizing variance but losing direct biological interpretability.
• Clinical Decision: Use feature selection when the goal is a diagnostic panel of specific measurable analytes. Use feature extraction when the goal is visualization or feeding into another predictive model.
• Example: Selecting the top 10 genes for a PCR test (selection) vs. using all genes to compute a risk score (extraction).

A linear dimensionality reduction technique that transforms data into a set of orthogonal principal components ordered by the variance they explain. In patient stratification, PCA is often the first step to decorrelate features and reduce noise before applying clustering algorithms. It works by computing the eigenvectors of the covariance matrix, projecting the original data onto a new coordinate system where the greatest variance lies on the first coordinate (PC1).

• Captures global linear structure
• Deterministic and computationally efficient
• Components are interpretable as weighted feature combinations

A non-linear dimensionality reduction algorithm specifically designed for visualizing high-dimensional data in 2D or 3D. t-SNE converts pairwise Euclidean distances into conditional probabilities representing similarities, then minimizes the Kullback-Leibler divergence between the high-dimensional and low-dimensional distributions. It excels at revealing local clusters in single-cell sequencing data but can distort global distances.

• Perplexity hyperparameter controls the balance between local and global structure
• Non-convex optimization produces different results per run
• Ideal for visual cluster validation, not feature extraction

A manifold learning technique that constructs a fuzzy topological representation of the data and optimizes a low-dimensional embedding to be as structurally similar as possible. UMAP better preserves both local and global data structure compared to t-SNE, making it the preferred tool for visualizing patient cohorts and single-cell populations. It scales efficiently to millions of data points.

• Based on Riemannian geometry and algebraic topology
• Preserves more global structure than t-SNE
• Faster runtime with n_neighbors controlling granularity

A neural network architecture where an encoder compresses input data into a low-dimensional latent space, and a decoder reconstructs the original input. The bottleneck layer forces the network to learn a compressed, meaningful representation. Variational autoencoders (VAEs) add a probabilistic constraint, making the latent space continuous and enabling generative sampling of new patient profiles.

• Non-linear generalization of PCA
• Deep Embedded Clustering (DEC) jointly optimizes latent space and cluster assignments
• Reconstruction error serves as an anomaly detection metric

Two distinct strategies for reducing dimensionality. Feature selection identifies a subset of the original variables (e.g., using LASSO regularization or mutual information), preserving interpretability crucial for biomarker identification. Feature extraction creates new composite variables (e.g., PCA components or autoencoder latent codes), which may capture more variance but sacrifice direct biological interpretability.

• Feature selection: Retains original feature semantics
• Feature extraction: Often yields better downstream clustering performance
• Elastic Net combines L1 and L2 penalties for robust selection in correlated genomic data

The process of estimating the true number of parameters needed to describe a dataset without significant information loss. Methods like Maximum Likelihood Estimation (MLE) on nearest-neighbor distances or the participation ratio based on eigenvalue spectra help determine the optimal target dimension. Over-reduction discards critical biological signal; under-reduction retains noise that degrades clustering.

• MLE approach estimates local intrinsic dimension per data point
• Helps set the number of PCA components or latent space size
• Critical for avoiding the curse of dimensionality in downstream models