Batch Effect Correction

The foundational approach to batch correction assumes technical variation can be modeled as a linear shift in high-dimensional space. Canonical Correlation Analysis (CCA) identifies maximally correlated linear combinations between batches to find a shared latent space. Seurat's integration method uses mutual nearest neighbors (MNNs) to identify corresponding cell populations across batches and applies a linear correction vector, known as anchoring, to align datasets. These methods are computationally efficient and interpretable but assume the batch effect is globally linear, which may fail for complex, non-linear distortions common in single-cell data.

Modern correction methods leverage deep learning to model complex, non-linear batch effects. Variational Autoencoders (VAEs) learn a probabilistic, batch-agnostic latent representation of cellular states. Architectures like scVI and scGen condition the decoder on batch labels, forcing the encoder to strip away technical noise. Adversarial training is also used, where a discriminator network tries to predict the batch of origin from the latent code, and the encoder is trained adversarially to fool it, resulting in a batch-invariant representation. These methods excel at capturing the complex distribution of single-cell data.

This framework views batch correction as a mathematical problem of aligning probability distributions. Optimal Transport (OT) finds the most efficient mapping from the distribution of cells in one batch to another, minimizing a cost function based on gene expression similarity. Tools like Mowgli and SCOT use OT to compute a coupling matrix that directly transforms cells from a source batch to a target batch. Unlike methods that find a shared low-dimensional space, OT performs a direct, cell-level alignment, making it highly interpretable and robust for preserving complex, rare cell populations without over-correction.

A robust heuristic for identifying equivalent biological states across batches. Two cells from different batches are considered mutual nearest neighbors if they are each other's nearest neighbor in a cross-batch comparison. This concept underpins the fastMNN and Scanorama algorithms. The key insight is that MNN pairs should represent the same cell type, and the difference in their expression vectors is a direct measurement of the batch effect. A correction vector is calculated from these pairs and applied to the entire dataset. This approach is highly effective for identifying shared cell types without assuming a global linear model.

The Harmony algorithm uses a novel iterative approach that integrates soft clustering with a mixture model. It first projects cells into a low-dimensional space via PCA, then iteratively clusters cells into diverse groups. For each cluster, it calculates a cell-specific correction factor based on the batch composition of that cluster. This correction is applied, and the process repeats until convergence. Harmony is exceptionally fast and scalable to massive datasets, and its soft-clustering approach allows it to handle complex experimental designs with multiple overlapping batch effects without over-correcting distinct cell types.

Validating batch correction requires a multi-faceted approach. Key metrics include:

• kBET (k-nearest neighbor Batch Effect Test): Quantifies the mixing of batches in local neighborhoods; a score of 1.0 indicates perfect mixing.
• ASW (Average Silhouette Width): Measures the compactness of cell-type clusters vs. batch clusters. A high ASW for cell type and low for batch indicates good correction.
• LISI (Local Inverse Simpson's Index): Separately measures the effective number of batches (iLISI) and cell types (cLISI) in a local neighborhood. Good correction maximizes cLISI and minimizes iLISI.
• Graph Connectivity: Assesses whether cells of the same type from different batches are connected in a k-nearest neighbor graph.