Attention Weights

Attention weights are computed via a softmax function, ensuring they are non-negative and sum to 1 for each query position. This allows them to be interpreted as a categorical probability distribution over input tokens.

• Each weight indicates the relative importance of a key position to a query.
• A weight near 0 means the position is effectively ignored.
• The sum of all weights for a single query is always exactly 1.

A weight is not an intrinsic property of a nucleotide but a dynamic, pairwise score computed between a query position and a key position. The same nucleotide can receive vastly different weights depending on the surrounding sequence context and the query it is being matched against.

• Captures long-range dependencies, such as enhancer-promoter interactions.
• The weight for position i attending to j is unique to that (i, j) pair in that specific input sequence.

Transformer layers use multiple independent attention heads that operate in parallel. Each head learns a distinct relational pattern, and their weights often specialize in different biological signals.

• One head may focus on splice donor sites, while another tracks open reading frames.
• Analyzing per-head weights reveals the model's decomposed reasoning strategy.
• A head that consistently attends to a specific motif is said to have learned a syntactic pattern.

In trained genomic models, attention distributions are often highly sparse, with the vast majority of weight concentrated on a few critical positions. This can be measured using entropy.

• Low entropy indicates focused, confident attention on a single motif.
• High entropy suggests the model is aggregating information broadly.
• Sparsity is a key metric for evaluating if a model has learned a crisp biological signal versus a diffuse, noisy one.

Attention weights across the depth of a model form a hierarchical feature extractor. Lower layers typically capture local, short-range patterns, while upper layers integrate long-range, abstract concepts.

• Early layers: Attention focuses on adjacent k-mers and local sequence composition.
• Middle layers: Attention links functional elements like transcription factor binding sites.
• Final layers: Attention integrates global context for the final prediction task.

The raw attention weight alone can be misleading. The true impact of a position is a product of the attention weight and the value vector's magnitude. A position with a high attention weight but a near-zero value vector contributes little to the output.

• Attention Rollout and Attention Flow are methods that combine weights across layers to estimate effective influence.
• This distinction is critical for avoiding false-positive biological interpretations from raw weight inspection alone.

The general class of methods that assign a relevance score to each input nucleotide or genomic region for a specific neural network prediction. Attention weights are one form of attribution, but other methods like gradient-based approaches provide different lenses into model logic.

• Quantifies the contribution of each input feature
• Essential for validating that models focus on biologically relevant signals
• Forms the foundation for regulatory compliance documentation

An axiomatic feature attribution method that computes the path integral of gradients from a baseline input to the actual input. Unlike raw attention weights, this method satisfies the completeness axiom, ensuring the sum of attributions equals the prediction difference.

• Provides theoretically grounded importance scores
• Uses a neutral baseline (e.g., all zeros) as reference
• Commonly used alongside attention for validation

A systematic perturbation technique that computationally mutates every nucleotide in a sequence to quantify its impact on model predictions. ISM provides a direct causal measure of importance that can be compared against attention weight patterns.

• Computationally intensive but model-agnostic
• Reveals causal nucleotide dependencies
• Gold standard for validating attention-based interpretations

The assignment of an importance score to each individual base pair in a genomic sequence. Attention weights naturally provide this granularity, but other methods like DeepLIFT and SHAP can also resolve contributions to single-nucleotide resolution.

• Highest possible resolution for interpretability
• Critical for identifying pathogenic single-nucleotide variants
• Enables precise motif boundary delineation

Quantitative measures that evaluate how accurately an attribution map—including attention weights—reflects the true decision-making logic of a genomic model. These metrics use perturbation experiments to test if high-attention regions are causally important.

• Includes metrics like ROAR and AOPC
• Distinguishes correlative from causal attention patterns
• Essential for regulatory-grade model validation

A method that clusters high-contribution genomic subsequences identified by attribution maps into recurring, biologically meaningful motif patterns. When applied to attention weight matrices, TF-MoDISco can reveal the sequence motifs that specific attention heads have learned to recognize.

• Bridges attribution scores to biological interpretation
• Identifies transcription factor binding site patterns
• Automates the discovery of learned sequence grammar