One-Hot Encoding

Each nucleotide is mapped to a linearly independent vector where exactly one element is '1' and all others are '0'. This ensures the model has no prior assumption about nucleotide similarity. The standard mapping is:

• A: [1, 0, 0, 0]
• C: [0, 1, 0, 0]
• G: [0, 0, 1, 0]
• T: [0, 0, 0, 1] This orthogonality guarantees that the Euclidean distance between any two distinct nucleotides is always √2, treating all substitutions as equally distinct before learning begins.

A sequence of length L is transformed into a binary matrix of shape (4 × L). Each column corresponds to a sequence position, and each row represents a nucleotide channel. This format is directly compatible with 1D convolutional neural networks (CNNs) , which treat the 4 channels as input features analogous to RGB channels in image processing. A convolutional filter of width k scanning this matrix learns position weight matrices (PWMs) that detect sequence motifs.

One-hot encoding produces an extremely sparse representation—for a sequence of length L, exactly L bits are '1' out of 4L total bits. While conceptually simple, this sparsity can be memory-intensive for large genomes. Storing the human genome (~3 billion base pairs) as a dense 32-bit float one-hot tensor requires approximately 48 GB of RAM. Production pipelines often use memory-mapped files or lazy loading to handle chromosome-scale sequences without exhausting system memory.

Unlike learned embeddings such as DNA2Vec or Nucleotide Transformer, one-hot encoding contains zero biological prior knowledge. A 'G' at position 5 and a 'G' at position 100 have identical representations with no context. This property is both a limitation—the model must learn all relationships from scratch—and a strength—it introduces no external bias, making it ideal as a raw input for architectures like Enformer or Basenji that learn regulatory syntax directly from sequence data.

IUPAC ambiguity codes (e.g., N for any base, R for A or G) cannot be represented by a single one-hot vector. The standard solution is to encode them as a probability distribution over the four canonical nucleotides:

• N: [0.25, 0.25, 0.25, 0.25]
• R: [0.50, 0.00, 0.50, 0.00]
• Y: [0.00, 0.50, 0.00, 0.50] This soft encoding preserves the 4-channel structure while accurately representing uncertainty at variable positions in consensus sequences.

A critical limitation: a sequence and its reverse complement produce entirely different one-hot matrices. The forward strand 'ATCG' and its reverse complement 'CGAT' have no inherent mathematical relationship in this representation. To enforce strand invariance, practitioners must either:

• Apply reverse complement augmentation during training, presenting both strands as independent samples.
• Use a siamese architecture with shared weights that processes both strands and averages the outputs.
• Pre-process data to use canonical k-mers before encoding.

Decomposes a sequence into all overlapping substrings of length k, then maps each k-mer to a unique integer or frequency vector. Unlike one-hot's single-nucleotide view, k-mer encoding captures local sequence context and motif-level patterns. The choice of k balances vocabulary size against information density—larger k values increase specificity but risk data sparsity.

A pre-trained embedding model that learns dense, distributed vector representations of variable-length k-mers. By applying the continuous bag-of-words (CBOW) and skip-gram algorithms to a corpus of non-overlapping genomic sequences, DNA2Vec captures semantic relationships where functionally similar k-mers occupy nearby positions in the latent space. This directly addresses one-hot encoding's inability to model nucleotide similarity.

A representation strategy that accounts for the double-stranded nature of DNA. Since a sequence and its reverse complement are biologically equivalent, this method ensures they map to identical or equivalent embedding vectors. Standard one-hot encoding treats the forward strand and its reverse complement as entirely distinct inputs, wasting model capacity on learning strand invariance from data alone.

A mechanism that injects information about the absolute or relative position of each token into the input embedding. One-hot encoded matrices contain no positional information—a nucleotide at index 0 is indistinguishable from one at index 1000. Positional encoding, using sinusoidal functions or learned embeddings, is essential for enabling transformer architectures to process sequential genomic data.

The IUPAC nucleotide notation (e.g., N for any base, R for purine, Y for pyrimidine) represents positions of uncertainty or natural variation in consensus sequences. Standard one-hot encoding cannot natively represent degenerate characters. Specialized embedding strategies must handle these ambiguous inputs, often by distributing probability mass across the possible bases or learning a separate embedding vector for each IUPAC code.

A tokenization strategy that segments coding sequences into non-overlapping triplets of nucleotides, directly aligning the vocabulary with the fundamental functional units of the genetic code. While one-hot encoding operates at the single-nucleotide level, codon tokenization creates a vocabulary of 64 tokens that captures the translation machinery's perspective, making it ideal for models predicting protein-coding potential or synonymous mutation effects.