Delta Scores

A delta score is formally defined as the arithmetic difference between two prediction outputs: Δ = P(alt) - P(ref). For a binary classifier predicting regulatory activity, a delta score of +0.45 indicates the alternate allele increases the predicted probability by 45 percentage points. In regression tasks predicting gene expression, the delta represents the absolute change in predicted transcript abundance. The computation requires exactly two forward passes through the model—one with the reference sequence and one with the alternate—making it computationally efficient for genome-wide variant scanning.

Delta scores rely on paired sequence design where the reference and alternate inputs differ only at the variant position. Key design considerations include:

• Flanking context length: Typically 500-1000bp surrounding the variant to capture regulatory elements
• Strand symmetry: Both forward and reverse complement sequences are often evaluated and averaged
• Padding strategy: Sequences shorter than the model's receptive field require zero-padding or dinucleotide shuffling
• Multi-allelic handling: For sites with more than two alleles, pairwise deltas are computed for each alternate against the reference

The sign and magnitude of a delta score encode distinct biological hypotheses:

• Positive delta: The alternate allele increases the predicted molecular phenotype (e.g., stronger transcription factor binding)
• Negative delta: The alternate allele decreases or ablates function (e.g., loss-of-function variants)
• Near-zero delta: The variant is predicted to be functionally neutral
• Magnitude thresholds: Empirical cutoffs (e.g., |Δ| > 0.1) are often calibrated against Deep Mutational Scan (DMS) ground truth data to classify variants as functionally significant

Delta scores are distinct from but complementary to feature attribution techniques:

• In-silico Mutagenesis (ISM) computes deltas for all possible single-nucleotide changes at every position, generating a mutation map
• Integrated Gradients and DeepLIFT attribute importance to reference nucleotides but do not directly quantify alternate allele effects
• SHAP values can approximate delta scores when the reference sequence serves as the background expectation
• Delta scores provide ground truth for attribution validation: A faithful attribution map should assign high importance to positions where mutations produce large deltas

Raw delta scores require calibration against experimental data to establish clinical or biological validity:

• Deep Mutational Scan (DMS) datasets provide high-throughput functional measurements for thousands of variants, serving as the gold standard benchmark
• ClinVar annotations enable calibration of delta score thresholds against pathogenic and benign classifications
• Population allele frequencies from gnomAD help distinguish tolerated common variants from rare deleterious ones
• Spearman correlation between delta scores and experimental measurements typically ranges from 0.4 to 0.7 for state-of-the-art genomic models

Computing delta scores across entire genomes requires optimized inference pipelines:

• Batched variant processing: Grouping variants by genomic region to reuse shared sequence context
• GPU-accelerated inference: Leveraging tensor parallelism for models with millions of parameters
• Pre-computed reference predictions: Caching reference allele scores to halve the required forward passes
• Variant Effect Predictors (VEPs) like Enformer and Basenji2 can score ~10,000 variants per second on modern hardware, enabling population-scale analyses

The foundational class of methods that assign a relevance score to each input nucleotide for a specific neural network prediction. Delta scores are a direct application of feature attribution, quantifying the difference in attribution between two alleles. Key approaches include gradient-based methods (Integrated Gradients, DeepLIFT), perturbation-based methods (in-silico mutagenesis), and Shapley value decompositions (SHAP). The choice of attribution method directly impacts the biological interpretability of the resulting delta score.

A systematic perturbation technique that computationally mutates every nucleotide in a sequence to quantify its impact on model predictions. ISM is the most direct method for computing delta scores: the prediction change between the reference and alternate allele is the delta score. While computationally expensive—requiring a forward pass for every possible substitution—ISM provides a model-agnostic, assumption-free measure of variant impact that serves as a gold standard for validating faster approximation methods.

Quantitative measures that evaluate how accurately an attribution map reflects the true decision-making logic of a genomic model. For delta scores to be trusted in clinical or regulatory contexts, the underlying attribution method must be faithful. Key metrics include:

• AOPC (Area Over the Perturbation Curve): measures prediction drop as salient nucleotides are removed
• ROAR (RemOve And Retrain): iteratively retrains the model after removing top-attributed features
• Infidelity Measure: quantifies the error between input perturbation and attribution perturbation

A high-throughput experimental method that assays the functional impact of thousands of genomic variants, often used as a ground-truth benchmark for validating delta scores. DMS data provides an empirical measurement of how each possible amino acid or nucleotide change affects a molecular phenotype—such as protein function or gene expression. Comparing computationally predicted delta scores against DMS measurements is the gold standard for assessing variant effect predictors.

A suite of tests designed to verify that an attribution method is sensitive to the learned parameters of the genomic model, not just the input data. The most critical test is model parameter randomization: if the model weights are scrambled but the attribution map remains unchanged, the method is not truly explaining the model. Delta scores derived from methods that fail sanity checks may reflect data artifacts rather than genuine learned biological signals.

The assignment of an importance score to each individual base pair in a genomic sequence, providing the highest possible resolution for model interpretability. Delta scores for single-nucleotide variants depend on accurate nucleotide-level attribution. Methods like Integrated Gradients and DeepLIFT produce per-nucleotide scores, while coarser methods like Grad-CAM yield region-level attributions that may obscure the precise contribution of a single base change.