Variant Quality Score Recalibration (VQSR)

VQSR constructs a Gaussian Mixture Model (GMM) to separate true variants from sequencing artifacts. The model fits a positive distribution on known true sites (e.g., HapMap, Omni) and a negative distribution on known false sites (e.g., strand bias artifacts, homopolymer errors). Each variant receives a Variant Quality Score Log Odds (VQSLOD) value—the log ratio of the probability that the variant belongs to the true distribution versus the false distribution. This transforms raw quality scores into well-calibrated error estimates.

VQSR evaluates each variant across multiple annotation dimensions simultaneously, moving beyond single-threshold hard filtering. Key features include:

• Mapping Quality Rank Sum Test (MQRankSum): Compares mapping qualities of reads supporting reference vs. alternate alleles
• Read Position Rank Sum Test (ReadPosRankSum): Detects positional bias where variants cluster near read ends
• Strand Odds Ratio (SOR): Identifies strand bias artifacts more sensitively than Fisher's exact test
• Base Quality Rank Sum Test (BaseQRankSum): Evaluates base quality differences between alleles
• Depth of Coverage (DP): Normalized read depth at the variant site

The GMM learns the covariance structure between these features, capturing complex artifact signatures that simple thresholding misses.

VQSR requires high-confidence truth resources to define the positive training distribution. Standard resources include:

• HapMap 3.3: High-confidence polymorphic sites from population studies
• 1000 Genomes Project Omni 2.5M: Genotyped sites with high validation rates
• Mills and 1000G Gold Standard Indels: Curated insertion/deletion truth sets
• dbSNP: Known variant catalog (used with caution, as it contains some false positives)

The model uses these resources to estimate the prior probability of true variation across the feature space, then applies this prior to recalibrate the discovery dataset.

VQSR outputs tranches—stratified sensitivity levels that allow users to select variant sets with specific false discovery rate profiles. Each tranche corresponds to a VQSLOD cutoff calibrated against the truth set:

• 99.9% tranche: Retains 99.9% of true variants, maximizing sensitivity for discovery
• 99.0% tranche: Balances sensitivity and specificity for clinical applications
• 90.0% tranche: Conservative filtering for applications requiring high specificity Users select a target truth sensitivity level, and VQSR applies the corresponding VQSLOD threshold to filter the discovery set. This provides principled false discovery rate control without arbitrary hard cutoffs.

VQSR builds separate GMMs for different variant classes because the annotation feature distributions differ substantially between:

• Single Nucleotide Polymorphisms (SNPs): Modeled with features like MQRankSum, ReadPosRankSum, and QD
• Insertions and Deletions (Indels): Modeled separately due to distinct error modes, including homopolymer length and tandem repeat context This stratification prevents the model from conflating SNP-specific artifacts with indel-specific errors, improving recalibration accuracy for both classes. Each model independently estimates the covariance structure of its annotation features.

VQSR differs fundamentally from hard filtering approaches that apply fixed annotation thresholds:

• Hard Filtering: Uses single-dimension cutoffs (e.g., QD < 2.0, FS > 60.0) applied independently, ignoring correlations between annotations
• VQSR: Models the joint distribution of all annotations simultaneously, capturing interactions like the relationship between strand bias and mapping quality
• Calibration: VQSR produces well-calibrated error probabilities, meaning a VQSLOD score of 3.0 corresponds to a true 1-in-1000 error rate, while hard-filtered quality scores often deviate from actual error rates For small datasets (< 30 exomes or < 10 genomes), hard filtering remains necessary because VQSR requires sufficient data to fit the GMM.

The core probabilistic engine of VQSR that models the distribution of variant annotation values. It fits a positive model on known true variants and a negative model on artifacts. Each variant's multidimensional annotation vector is evaluated against these distributions to assign a new, well-calibrated Variant Quality Score Log Odds (VQSLOD) score that reflects the true probability of being a real variant versus a sequencing artifact.

A critical preprocessing step applied before VQSR. BQSR uses a covariate model to detect and correct systematic errors in the per-base quality scores emitted by the sequencing instrument. It analyzes patterns across:

• Read group (sequencing lane)
• Machine cycle (position in read)
• Dinucleotide context (preceding and current base) Without BQSR, the raw quality scores fed into VQSR would contain systematic biases, degrading the accuracy of the recalibrated variant probabilities.

The final output metric of VQSR, representing the log odds ratio of a variant being real under the positive Gaussian mixture model versus being an artifact under the negative model. A VQSLOD of 0 means equal probability; positive values indicate likely true variants. The tranche sensitivity threshold (e.g., 99.7%) defines the VQSLOD cutoff where a specified percentage of true variants from the training set are retained, allowing users to balance precision and recall for their specific application.

The multidimensional input vector that VQSR evaluates for each variant. Standard annotations include:

• QualByDepth (QD): Variant confidence normalized by read depth
• StrandOddsRatio (SOR): Strand bias metric improved over Fisher's exact test
• MappingQualityRankSumTest (MQRankSum): Compares mapping qualities of reference vs. alternate allele reads
• ReadPosRankSumTest (ReadPosRankSum): Evaluates positional bias of alternate alleles within reads
• FisherStrand (FS): Phred-scaled probability of strand bias These features collectively distinguish true biological variation from systematic sequencing artifacts.

Two competing approaches for variant filtration:

• Hard Filtering: Applies fixed, user-defined thresholds to individual annotation values (e.g., QD < 2.0 → fail). Simple but fails to capture complex, multivariate relationships between annotations.
• VQSR: Uses a Gaussian mixture model to learn the multidimensional annotation profile of true variants, assigning a single calibrated probability. Superior for high-depth whole genomes and large cohorts where training data is abundant. Hard filtering remains useful for small datasets or non-model organisms lacking truth sets, where VQSR cannot be trained effectively.