Highly Variable Genes (HVG)

HVG selection explicitly models the dependency between gene expression mean and variance. In scRNA-seq, variance naturally increases with mean expression due to Poisson sampling noise. Algorithms fit a loess regression or gamma-Poisson GLM to the log(variance) vs log(mean) trend. Genes with positive residuals—variance significantly exceeding the fitted trend—are flagged as HVGs. This prevents the selection of highly expressed but uninformative housekeeping genes.

To rank genes, methods compute a standardized variance that corrects for the mean-dependence. Common approaches include:

• vst (Variance Stabilizing Transformation): Applies a transformation to render variance independent of mean, then ranks by transformed variance.
• dispersion (cv2): Uses the squared coefficient of variation, comparing observed dispersion to the expected sampling noise.
• deviance: Calculates the deviance of a fitted null model, capturing excess variability beyond technical expectations.

To avoid global trend distortions, genes are binned by average expression into equal-frequency groups. Within each bin, the median absolute deviation (MAD) or z-score of variance is calculated. This local normalization ensures that HVG selection is not biased toward high-expression regimes. A gene is selected if its variance exceeds a threshold (e.g., z-score > 0.5) relative to its local bin neighbors.

Most workflows select a fixed number of top HVGs (e.g., top 2,000 or top 5,000) rather than using a strict statistical cutoff. This heuristic balances computational tractability with biological signal retention. Selecting too few genes risks missing rare cell-type markers; selecting too many introduces noise that degrades Principal Component Analysis (PCA) and graph-based clustering performance.

In multi-sample studies, naive HVG selection can capture batch effects rather than biology. Advanced methods like scran or Seurat's SelectIntegrationFeatures identify genes that are highly variable consistently across batches. This involves ranking genes by variance within each batch independently, then selecting those with high median rank across all batches, ensuring the selected features represent robust biological heterogeneity.

Different toolkits implement distinct HVG strategies:

• Seurat v5: Defaults to vst, selecting 2,000 features by default. It fits a loess curve to log(variance) vs log(mean) and ranks by standardized variance.
• Scanpy: Offers seurat, seurat_v3, and cell_ranger flavors. The seurat_v3 method uses a count-based gamma-Poisson model without log-transformation, directly estimating dispersion parameters.
• scran: Models technical noise using spike-in RNA or by decomposing variance into biological and technical components via a mixed-effects model.

The process of scaling raw count data to adjust for differences in sequencing depth and capture efficiency between cells. Without proper normalization, the variance calculated during HVG selection would be dominated by technical artifacts rather than biological heterogeneity. Common methods include library-size normalization (counts per million) and more sophisticated approaches like SCTransform, which uses regularized negative binomial regression to stabilize variance while preserving biological signal.

Non-biological systematic variation introduced by technical factors like different experimental runs, reagents, or sequencing lanes. If uncorrected, batch effects can dominate the variance structure, causing HVG selection to identify batch-specific genes rather than biologically informative features. This is why HVG selection is often performed after data integration or within individual batches before merging.

The mathematical transformation of high-dimensional single-cell data into a lower-dimensional space, and the primary consumer of HVG selection. By restricting analysis to the top 2,000–5,000 most variable genes, Principal Component Analysis (PCA) can focus on the axes of greatest biological variation. This compressed representation then feeds into t-SNE or UMAP for visualization and graph-based clustering for cell type discovery.

A sparse numerical matrix where rows represent genes and columns represent cell barcodes, storing the number of transcripts detected per gene in each cell. HVG selection operates directly on this matrix after quality control (QC) filtering. The sparsity of single-cell data—where over 90% of entries are zeros due to dropout events—necessitates variance-stabilizing transformations before meaningful HVG identification can occur.