Highly Variable Gene Selection

HVG selection corrects for the inherent dependency between a gene's mean expression and its variance. In single-cell data, highly expressed genes naturally show higher absolute variance. Methods like variance-stabilizing transformation (VST) or deviance-based models fit a non-linear curve to this relationship, then select genes that deviate significantly above the expected variance for their expression level. This prevents the selection of highly expressed but uninformative housekeeping genes.

The core objective is to isolate biological heterogeneity from technical noise. Technical variation arises from:

• Sampling noise: Low capture efficiency in droplet-based systems
• Amplification bias: Uneven PCR amplification across transcripts
• Dropout events: Genes with zero counts due to failed detection

HVG methods model the expected technical variance and select genes where observed variance exceeds this baseline, enriching for genuine biological signal.

Three dominant approaches are implemented in standard workflows:

• Seurat v3/v5 VST: Applies variance-stabilizing transformation, bins genes by mean expression, and selects top variance within each bin. Returns 2,000 HVGs by default.
• scran modelGeneVar: Fits a gamma-Poisson GLM to decompose total variance into biological and technical components, selecting genes with positive biological variance.
• Scanpy highly_variable_genes: Implements both dispersion-based (normalized variance over mean) and Pearson residual methods for HVG identification.

HVG selection directly shapes the latent space used for clustering and visualization:

• PCA input: Only HVGs are used for principal component computation, ensuring PCs capture biological rather than technical variation.
• Graph construction: K-nearest neighbor graphs built on HVG-derived PCs produce more biologically coherent Leiden clusters.
• UMAP/t-SNE: Dimensionality reduction embeddings reflect true cell-type relationships when built from HVG-filtered data.
• Marker gene discovery: Differential expression testing on HVGs reduces multiple-testing burden and enriches for discriminative features.

The number of HVGs selected is a critical hyperparameter:

• Default range: 1,000–5,000 genes, typically 2,000 for 10x Genomics data
• Too few HVGs: Risk excluding rare cell-type-specific genes and subtle biological gradients
• Too many HVGs: Introduce noise, dilute biological signal in PCA, and increase computational cost
• Sparsity consideration: In highly sparse datasets (e.g., Smart-seq2 with low dropout), dispersion-based methods may require adjusted thresholds to avoid selecting genes driven by zero-inflation artifacts.

HVG selection behavior depends critically on upstream normalization:

• Seurat 'vst': Operates on log-normalized data; robust for UMI-based technologies
• Seurat 'mean.var.plot': Legacy dispersion-based method on normalized data
• Scanpy 'seurat_v3': Expects raw counts and applies the count-based VST internally
• Pearson residuals: Applied to raw counts with regularized negative binomial regression, bypassing explicit normalization

Choosing the wrong flavor for your normalization state produces misleading variance estimates and suboptimal gene sets.

A linear dimensionality reduction algorithm that transforms high-dimensional gene expression data into a set of orthogonal principal components capturing the maximum variance. Highly variable gene selection is a critical preprocessing step before PCA to ensure that the components reflect biological heterogeneity rather than technical noise. PCA is often used to verify that selected HVGs capture the dominant axes of variation in the dataset.

A nonlinear dimensionality reduction technique that preserves both local and global data structure, widely used to visualize single-cell transcriptomic landscapes. UMAP operates on the principal component space derived from highly variable genes, making HVG selection a foundational step that directly impacts the quality and biological interpretability of the final visualization.

A preprocessing step that adjusts raw single-cell gene expression counts to account for differences in sequencing depth and capture efficiency. Normalization must be performed before highly variable gene selection, as unnormalized data would cause HVG algorithms to select genes driven by technical artifacts rather than genuine biological variation. Common methods include library-size normalization and regularized negative binomial regression.

A computational process that removes technical variation introduced by different experimental batches. If batch effects are not corrected before HVG selection, the algorithm may select genes that distinguish batches rather than cell types. Integration methods like Harmony or scVI are often applied post-normalization but can influence which genes are identified as highly variable.

A computational method that assigns each cell a numerical score representing its progression through the cell cycle phases. Cell cycle genes are often highly variable and can dominate HVG selection, obscuring more biologically interesting signals. Practitioners frequently regress out cell cycle scores or explicitly exclude known cell cycle genes from the HVG list to focus on differentiation-related variation.

A deep generative model based on a variational autoencoder that learns a probabilistic latent representation of gene expression. scVI explicitly models batch effects and zero-inflation while identifying biologically meaningful variation. Unlike traditional HVG selection, scVI operates on the full gene set and learns a latent space that implicitly performs feature selection, often outperforming mean-variance-based HVG methods.