K-mer Encoding

A window of length k slides across the DNA sequence one nucleotide at a time, extracting every possible contiguous substring. For a sequence of length L, this generates exactly L - k + 1 overlapping k-mers.

• Example: Sequence ATCG with k=3 produces ATC and TCG.
• Granularity: Smaller k values capture local nucleotide composition; larger k values capture specific motifs.
• Stride: Typically 1, but can be increased to reduce feature dimensionality.

Each unique k-mer is mapped to an index in a fixed-size vector, where the value represents the count or frequency of that k-mer in the sequence.

• Vocabulary Size: 4^k possible k-mers (e.g., k=4 yields 256 dimensions).
• Sparsity: Most vectors are sparse, as a single sequence contains only a fraction of all possible k-mers.
• Normalization: Counts are often normalized by sequence length or transformed via TF-IDF to down-weight common genomic repeats.

Rather than a frequency vector, the sequence can be represented as a list of integer IDs, where each k-mer is assigned a unique numerical token. This preserves the sequential order for downstream models like transformers.

• Vocabulary Construction: A dictionary maps each k-mer string to an integer (e.g., AAA → 0, AAC → 1).
• Sequence Compression: A 1000bp sequence becomes a sequence of 997 integer tokens (for k=4).
• Unknown Handling: K-mers containing ambiguous bases (e.g., N) are either mapped to a special [UNK] token or split into constituent known k-mers.

To account for the double-stranded nature of DNA, a k-mer and its reverse complement are often collapsed into a single canonical representation. This ensures that a gene on the forward strand and its reverse complement on the opposite strand map to the same feature.

• Lexicographic Selection: The alphabetically smaller of the k-mer and its reverse complement is chosen.
• Dimensionality Reduction: Halves the effective vocabulary size.
• Strand Invariance: Critical for tasks where the functional element's orientation is unknown or irrelevant.

The parameter k controls the trade-off between resolution and generalization. No single k is universally optimal; the choice depends on the biological signal and downstream task.

• Small k (3-4): Captures codon usage and GC content; computationally cheap but low specificity.
• Medium k (5-7): Balances motif resolution with manageable vocabulary size; common for regulatory element prediction.
• Large k (8-12): Captures specific transcription factor binding sites; risks extreme sparsity and overfitting.
• Mismatch K-mers: A variant allowing a defined number of mismatches can increase robustness to sequencing errors.

K-mer encoding is a higher-order generalization of one-hot encoding. While one-hot encodes single nucleotides independently, k-mer encoding captures local n-gram context.

• One-Hot: A 4-dimensional vector per position; no sequential context.
• K-mer Frequency: A 4^k-dimensional vector per sequence; captures local composition but loses positional order.
• K-mer Tokenization: A sequence of integer tokens; preserves both local context and positional order for models like DNABERT.

A sparse binary representation where each nucleotide (A, C, G, T) is mapped to a distinct orthogonal basis vector, creating a 4-channel matrix that serves as a raw input for convolutional neural networks. Unlike k-mer encoding, which captures local sequence motifs, one-hot encoding provides a single-nucleotide resolution view of the sequence. The resulting matrix has dimensions [4 x sequence_length], where each column contains a single '1' and three '0's. This representation is often the first layer in genomic deep learning pipelines before learned embeddings are extracted.

A standardized representation that selects the lexicographically smaller of a k-mer and its reverse complement to collapse the strand-specific sequence space into a single, unambiguous feature. For example, the k-mer 'AAGCT' and its reverse complement 'AGCTT' are both mapped to the same canonical identifier. This strand-invariant encoding reduces the vocabulary size by nearly half and prevents the model from learning spurious strand-specific patterns, which is critical because DNA is double-stranded and regulatory elements function regardless of orientation.

A data compression algorithm repurposed for genomic tokenization that iteratively merges the most frequent adjacent nucleotide pairs into subword units. BPE builds a vocabulary that balances sequence granularity with context length, creating tokens of variable length rather than fixed k. This approach, used by models like DNABERT-2, allows the tokenizer to learn that common motifs like 'TATA' or 'CG' should be treated as single tokens, while rare combinations remain as individual nucleotides. The result is a more efficient encoding that captures biological syntax.

A domain-specific data augmentation technique that doubles the training dataset by presenting both the forward strand and its reverse complement to the model. This enforces strand-invariance in the learned genomic embeddings, reflecting the biological reality that regulatory proteins can bind to either strand. During training, each sequence is randomly presented in its forward or reverse complement orientation, effectively teaching the model that the functional meaning of a sequence is independent of which strand it resides on.

A tokenization strategy that segments an mRNA transcript or coding DNA sequence into non-overlapping triplets of nucleotides, directly aligning the vocabulary with the fundamental functional units of the genetic code. Each codon corresponds to a specific amino acid or stop signal, making this representation inherently biologically meaningful. Unlike arbitrary k-mer encoding, codon tokenization preserves the translational reading frame and captures the functional consequences of nucleotide substitutions at the protein level.

An architecture that learns genomic representations at multiple resolutions simultaneously—such as nucleotide, codon, and gene level—by aggregating features from different convolutional kernel sizes or transformer layers. This approach addresses the limitation that a single k-mer size cannot capture both local motifs (e.g., transcription factor binding sites of 6-12 bp) and long-range regulatory syntax (e.g., enhancer-promoter interactions spanning kilobases). The model produces a hierarchical embedding that encodes biological information across scales.