Harmony

Harmony uses a soft k-means clustering approach that assigns cells to multiple clusters probabilistically rather than making hard assignments. This allows cells to belong to a cell-type cluster and a dataset-specific cluster simultaneously.

• Penalizes clusters that are dominated by a single dataset
• Encourages mixing of cells from different batches within each cluster
• The soft assignment matrix is used to compute a maximum diversity clustering

The algorithm applies a diversity penalty to the soft clustering objective. For each cluster, Harmony calculates the proportion of cells from each dataset and penalizes clusters where one dataset dominates.

• Uses an entropy-based regularization term
• The penalty strength is controlled by a parameter theta (θ)
• High theta forces aggressive mixing; low theta preserves biological structure
• This prevents the algorithm from simply grouping cells by batch

Once cells are softly clustered, Harmony estimates a dataset-specific linear correction factor for each cluster. The correction is computed as the difference between the cluster centroid and the dataset-specific centroid within that cluster.

• Correction is a simple shift vector applied to each cell
• Preserves the relative distances between cells within a cluster
• Avoids the overcorrection problems of global alignment methods
• Correction is applied iteratively until convergence

Harmony operates in an expectation-maximization (EM) framework that alternates between two steps until convergence:

1. E-step: Softly assign cells to clusters based on current corrected coordinates
2. M-step: Re-estimate cluster centroids and dataset-specific correction factors

• Typically converges in 5-10 iterations
• The algorithm is deterministic given the same initialization
• Output is a batch-corrected embedding matrix suitable for downstream analysis

Harmony's performance is evaluated using quantitative metrics that assess both batch mixing and biological preservation:

• Local Inverse Simpson's Index (LISI): Measures the effective number of datasets in each cell's local neighborhood. A high integration LISI (iLISI) indicates good mixing
• Average Silhouette Width (ASW): Evaluates whether cells of the same type remain close together after correction
• kBET acceptance rate: Tests whether the local batch distribution matches the global distribution
• Harmony consistently achieves near-1.0 kBET acceptance rates on benchmark datasets

Harmony is designed for large-scale single-cell datasets with hundreds of thousands to millions of cells. Its linear-time complexity makes it suitable for atlas-scale integration projects.

• Runtime scales linearly O(N) with the number of cells
• Processes 100,000 cells in under 2 minutes on standard hardware
• Implemented in both R (harmony package) and Python (harmonypy)
• Integrates seamlessly with Seurat, Scanpy, and SingleCellExperiment workflows
• Memory-efficient implementation uses sparse matrix operations

A systematic non-biological source of variation introduced across different experimental batches—such as different processing dates, reagents, or technicians—that can confound downstream analysis. In single-cell RNA sequencing, batch effects manifest as global shifts in gene expression distributions that obscure true biological differences. If uncorrected, they can lead to spurious clustering where cells group by batch rather than by cell type, producing false discoveries in differential expression and trajectory analyses.

A batch correction method that identifies pairs of cells from different batches that are mutual nearest neighbors in high-dimensional expression space. These MNN pairs are assumed to represent the same biological cell type and are used to estimate a correction vector that is then applied to all cells. The method is robust to differences in cell-type composition between batches and does not require a shared gene set, making it suitable for cross-platform integration scenarios.

A diversity score computed for each cell's local neighborhood that quantifies the effective number of different batches (iLISI) or cell types (cLISI) present. An iLISI score approaching the total number of batches indicates perfect mixing, while a cLISI score of 1.0 indicates that a neighborhood contains only one cell type. Used together, these metrics provide a dual assessment of integration quality—confirming that batches are mixed while cell-type purity is preserved.

The process of evaluating whether a batch correction method has removed true biological variation alongside technical noise. Overcorrection occurs when algorithms aggressively align cells from different batches, erasing meaningful biological distinctions such as rare cell populations or subtle activation states. Assessment typically involves measuring the preservation of known cell-type clusters, the variance explained by biological covariates, and the recovery of expected marker gene expression patterns post-correction.

A critical experimental design flaw where the batch variable is perfectly correlated with the biological condition of interest. For example, if all control samples are processed on Monday and all treatment samples on Tuesday, it becomes statistically impossible to separate technical artifacts from the true biological signal. No computational method can fully rescue a confounded design; the only solution is proper randomization of conditions across batches during experimental planning.

Systematic technical variation that remains in a dataset after an initial batch correction procedure has been applied. Residual effects often indicate incomplete modeling of the experimental design—such as unaccounted-for covariates like sample processing time or reagent lot numbers. Detection typically involves visualizing post-correction data with dimensionality reduction and applying quantitative metrics like kBET; significant residual effects may require a secondary, post-hoc correction step or revisiting the original model specification.