Token Smuggling

Attackers exploit differences in Unicode normalization forms (NFC vs NFD) to hide malicious instructions. A string that appears safe after sanitizer normalization may decompose into dangerous tokens when the model's tokenizer processes it.

• Mechanism: The sanitizer normalizes to NFC, but the tokenizer reads raw bytes or a different form
• Example: A combined character like é (single codepoint) passes sanitization, but the tokenizer splits it into e + ´, altering downstream token boundaries
• Impact: Commands like IGNORE_PREVIOUS can be reconstructed from fragments that individually appear harmless

Tokenizers often treat whitespace as a prefix indicator for new tokens. Attackers insert or remove spaces to force the tokenizer to merge or split words in ways the sanitizer did not anticipate.

• Mechanism: The sanitizer sees DROP TABLE as two distinct words, but removing the space causes the tokenizer to produce a single, unexpected token
• Example: SYSTEM:IGNORE without a space may tokenize as a single control-like token that triggers override behavior
• Defense: Normalize all whitespace to a canonical form before both sanitization and tokenization

Modern tokenizers use subword algorithms like Byte-Pair Encoding (BPE). Attackers craft inputs where malicious commands are split across multiple benign subword tokens that the model reassembles during inference.

• Mechanism: The word DISREGARD might tokenize as DIS + REG + ARD. A filter scanning for DISREGARD as a complete token misses it entirely
• Example: _assistant and _override pass separately but concatenate in the model's attention context to form a command
• Key Insight: Sanitizers operating on raw text cannot see the token boundaries the model will ultimately use

Tokenizers reserve special tokens for control functions (e.g., <|endoftext|>, <|im_start|>). If user input containing these raw strings is not properly escaped, attackers can inject synthetic control tokens that alter the model's parsing state.

• Mechanism: Input containing the literal string <|im_start|>system may be tokenized as an actual control token, not as text
• Example: An attacker injects <|im_end|> to prematurely terminate the system prompt context, then follows with new instructions
• Defense: Always escape or strip special token literals from user input before tokenization

When a system uses one tokenizer for content filtering and a different tokenizer for the actual language model, attackers exploit the boundary disagreements between them.

• Mechanism: A moderation API uses a BERT-based tokenizer, but the generation model uses a GPT-family tokenizer with different vocabulary and merge rules
• Example: A phrase that tokenizes as 3 safe tokens in the filter tokenizer becomes 2 tokens in the model tokenizer, one of which matches a blocked command
• Best Practice: Use identical tokenization pipelines for both filtering and generation

Byte-level BPE tokenizers can represent any Unicode character as a sequence of byte tokens. Attackers encode malicious payloads as raw byte sequences that bypass character-level filters but are decoded by the tokenizer into executable instructions.

• Mechanism: The string IGNORE encoded as individual byte tokens (I, G, N, O, R, E) passes filters looking for the complete word
• Example: Using zero-width joiners or bidirectional text markers to force unexpected byte-token boundaries
• Impact: Filters that operate at the character or word level are blind to byte-level token reconstruction

The foundational defense is ensuring the tokenizer used by the security filter is byte-for-byte identical to the model's tokenizer. Attackers exploit mismatches where a filter sees [harmless] but the model's tokenizer splits the string differently, reconstructing a malicious command. BPE (Byte-Pair Encoding) merging rules must be mirrored exactly. - Align pre-tokenization regex patterns - Match special token handling (e.g., <|endoftext|>) - Use the model's native tiktoken or sentencepiece library directly in the guard layer

Smuggling often uses homoglyphs and multi-byte Unicode sequences that look identical to ASCII but have different token boundaries. Defenses must apply NFKC normalization to collapse compatibility characters before tokenization. - Strip zero-width characters (U+200B, U+200C, U+200D) - Normalize fullwidth Latin (ＦＵＬＬＷＩＤＴＨ) to standard ASCII - Reject or normalize bidirectional text override characters that reverse string rendering order - Convert all text to a canonical form before any filtering logic executes

Instead of scanning raw text, this defense tokenizes the input and analyzes the resulting token ID sequence directly. Malicious payloads hidden across token boundaries become visible when inspecting the integer token stream. - Scan for known dangerous token sequences in embedding space - Detect anomalous token adjacency patterns that deviate from natural language - Flag inputs where a single 'safe' word tokenizes into sub-tokens that independently form dangerous commands - Implement a token-level intrusion detection system that operates on the model's native vocabulary

A defense-in-depth technique that decodes the input back to text after initial tokenization, then re-tokenizes it with a different algorithm. Discrepancies between the two token sequences indicate smuggling attempts. - Use a secondary tokenizer (e.g., WordPiece vs BPE) as a consistency check - Compare token counts and boundary positions between passes - Flag inputs where reconstruction produces a different byte sequence than the original - This catches attacks that exploit implementation-specific quirks in a single tokenizer

Token smuggling often relies on creating extremely long, artificially constructed tokens that span what should be multiple logical units. Enforcing a strict maximum token length in bytes prevents attackers from packing entire commands into a single token. - Reject any single token exceeding a configurable byte threshold - Monitor for inputs generating tokens with abnormally high byte-per-token ratios - This is particularly effective against attacks that exploit rare Unicode codepoints to create oversized BPE merges

Beyond token-level inspection, this defense computes the embedding vector of the input and compares it to the embedding of its sanitized version. A smuggled payload that changes the semantic meaning will produce a large cosine distance between the two vectors. - Generate embeddings using the target model's embedding layer - Flag inputs where sanitization produces a significantly different vector - This catches attacks where the surface text appears benign but the underlying token structure encodes malicious intent - Combine with a similarity threshold tuned to your application's tolerance