Data Integration

The primary goal of integration is to remove systematic, non-biological variation introduced by technical factors such as different experimental runs, reagents, or sequencing lanes. Algorithms learn a correction vector that aligns datasets in a shared latent space.

• Linear methods (e.g., ComBat, limma) model batch effects as additive/multiplicative shifts
• Deep learning methods (e.g., scVI, trVAE) use variational autoencoders to learn non-linear corrections
• Mutual nearest neighbors (MNN) identifies matching cell pairs across batches to estimate the correction
• Failure to correct batch effects leads to false discoveries in differential expression and spurious clustering driven by technical artifacts

A successful integration must not over-correct and erase genuine biological signals. The distinction between wanted variation (cell type, disease state, developmental stage) and unwanted variation (batch, donor, protocol) is context-dependent.

• k-nearest-neighbor batch effect test (kBET) quantifies local batch mixing without assuming discrete clusters
• Average silhouette width (ASW) measures how well integrated cells maintain their cell-type identity
• Local inverse Simpson's index (LISI) evaluates the diversity of batch labels in a cell's local neighborhood
• Over-correction manifests as homogeneous embeddings where distinct cell types are artificially merged

Many state-of-the-art methods identify anchors—cells in similar biological states across datasets—to guide the alignment process. These mutual nearest neighbors serve as reference points for computing the transformation that maps one dataset onto another.

• Seurat v3+ uses canonical correlation analysis (CCA) to find anchors in a shared correlation space
• scAlign learns an encoder-decoder mapping between datasets using anchor pairs
• Harmony iteratively clusters cells and applies soft-clustering-based correction without explicit anchors
• Anchor-based methods scale well to large atlases with millions of cells by subsampling representative reference cells

Modern integration methods must handle datasets with millions of cells spanning dozens of batches. Computational efficiency is achieved through approximate nearest neighbor search, out-of-memory computation, and subsampling strategies.

• scVI uses stochastic variational inference with mini-batching to train on arbitrary numbers of cells
• BBKNN replaces the full k-nearest neighbor graph with a batch-balanced variant computed per batch
• Online iNMF (used in LIGER) processes data incrementally without loading the full matrix into RAM
• GPU acceleration via RAPIDS and PyTorch enables sub-hour integration of million-cell datasets

Advanced integration extends beyond aligning multiple scRNA-seq datasets to fusing different data modalities and even cross-species comparisons. This requires mapping features into a shared latent space where correspondences can be established.

• Multimodal integration combines scRNA-seq with scATAC-seq, CITE-seq protein data, or spatial transcriptomics
• Cross-species integration maps homologous genes to compare cell types between mouse, human, and other model organisms
• TotalVI jointly models RNA and protein counts in CITE-seq data using a single probabilistic framework
• SATURN uses protein language models to map genes across evolutionarily distant species without relying on one-to-one orthology

Rather than integrating all datasets simultaneously, reference mapping projects new query datasets onto a pre-built, well-annotated reference atlas. This enables rapid annotation and avoids recomputing the full integration with each new sample.

• Symphony builds a compressed reference from a Harmony-integrated atlas for rapid query mapping
• scArches fine-tunes a pre-trained scVI or trVAE model on new data with minimal retraining
• Azimuth provides a web-based interface for mapping human PBMC and motor cortex queries to reference atlases
• Reference mapping is essential for clinical deployment where new patient samples must be analyzed against a standardized reference

The primary adversary that data integration algorithms seek to overcome. Batch effects are non-biological systematic variations introduced by technical factors—different experimental runs, reagents, sequencing lanes, or technicians—that confound true biological signal. Without correction, cells cluster by technical origin rather than cell type. Integration methods model and remove these effects while preserving biological variability across conditions or disease states.

A foundational concept in single-cell integration where cells from different batches are considered corresponding if they are mutual nearest neighbors in a shared feature space. MNN pairs identify overlapping biological populations across datasets, defining anchor points for batch correction. This principle underpins algorithms like fastMNN and Scanorama, which use these anchors to estimate and subtract batch-specific variation vectors without assuming a global linear correction model.

A statistical dimensionality reduction technique adapted for single-cell integration to identify shared correlation structures between datasets. Unlike PCA, which finds variance within a single matrix, CCA finds linear combinations of features that maximize correlation across two datasets. In the Seurat integration workflow, CCA identifies a conserved biological subspace, enabling the alignment of cells from different conditions into a common reference before downstream clustering and differential expression analysis.

An iterative, soft-clustering algorithm that integrates single-cell data by correcting principal component embeddings. Harmony alternates between clustering cells into diverse groups and applying dataset-specific linear correction factors to each cluster centroid. This approach is computationally efficient, scales to large atlases, and does not require a designated reference dataset. It is particularly effective for integrating complex experimental designs with multiple overlapping batch variables.

A deep generative model that uses variational autoencoders to learn a latent representation of single-cell data while explicitly modeling batch effects. scVI treats gene expression counts as samples from a zero-inflated negative binomial distribution conditioned on batch annotations and latent biological variables. This probabilistic framework provides uncertainty estimates, supports differential expression testing, and enables scArches for reference mapping—projecting new query cells onto an existing integrated atlas without retraining.

Quantitative measures to assess whether integration successfully mixed batches while preserving biological structure. Key metrics include:

• kBET: Tests if the local batch label distribution matches the global distribution, measuring mixing quality.
• ASW (Average Silhouette Width): Evaluates whether cells cluster by cell type rather than batch after integration.
• LISI (Local Inverse Simpson's Index): Separately quantifies batch mixing (iLISI) and cell-type separation (cLISI) in local neighborhoods.
• Graph connectivity: Measures whether cells of the same type from different batches are connected in a nearest-neighbor graph.