Token Smuggling

Token smuggling leverages BPE tokenization quirks to hide malicious strings. By splitting a forbidden word across token boundaries—for example, encoding 'ignore' as 'ign' + 'ore'—the concatenated text is legible to the LLM but invisible to pattern-matching filters that scan for the complete token. Attackers map a model's specific vocabulary to identify cross-token boundary vulnerabilities where dangerous instructions can be fragmented and reassembled at inference time.

Content safety filters typically operate on pre-tokenized text or complete token sequences. Token smuggling bypasses these by constructing payloads where prohibited terms are split into innocuous subword units. A filter scanning for 'delete all files' will miss it when encoded as 'del' + 'ete all fi' + 'les'. The LLM's autoregressive decoding process naturally reassembles these fragments, executing the malicious instruction while the filter sees only benign subword tokens.

Attackers exploit Unicode equivalence to craft payloads that tokenize differently than they render. Using homoglyphs, zero-width characters, or confusable Unicode sequences, a string that visually reads as a malicious command may tokenize into entirely different token IDs. For example, replacing Latin 'a' with Cyrillic 'а' (U+0430) produces a visually identical character that maps to a distinct token, allowing the payload to evade vocabulary-based denylists while the model interprets the semantic intent correctly.

Token smuggling is often chained with context window poisoning to maximize impact. An attacker first floods the context with benign padding tokens to exhaust attention on safety instructions, then inserts a smuggled payload fragmented across token boundaries. The combination ensures that: (1) the safety system prompt is attentionally diluted, and (2) the malicious instruction is reassembled by the model without triggering mid-context filters. This dual-stage approach is particularly effective against agents with large context windows.

Effective token smuggling requires knowledge of the target tokenizer. Different models use different BPE vocabularies—GPT-4's cl100k_base differs from Llama's SentencePiece tokenizer. An attacker reverse-engineers the vocabulary to find minimal fragment pairs that, when concatenated, form dangerous instructions. Tools like tiktoken and HuggingFace tokenizers are used to probe token boundaries and identify exploitable splits. This reconnaissance phase is critical: a payload crafted for one tokenizer will likely fail against another.

Defending against token smuggling requires semantic-level filtering rather than surface-level pattern matching. Effective mitigations include: re-tokenizing inputs with multiple tokenizers to detect boundary anomalies, using perplexity-based detection to flag unusual token sequences, and deploying LLM-based content classifiers that evaluate the reassembled semantic meaning rather than raw token streams. Input sanitization that normalizes Unicode and strips zero-width characters before tokenization also closes common smuggling vectors.

An exploit that targets document segmentation strategies in RAG pipelines. By placing malicious content precisely at chunk boundaries, attackers ensure their payload is retrieved as a standalone, authoritative context fragment rather than being diluted by surrounding benign text.

• Exploits recursive character text splitting algorithms
• Malicious content becomes a self-contained retrieval unit
• Bypasses contextual integrity checks that rely on surrounding text
• Particularly effective against fixed-size chunking with no overlap

The manipulation of an agent's recursive summarization process to cause critical safety instructions to be dropped or distorted as the context is compressed over multiple turns. Attackers inject content designed to survive or corrupt the summarization step.

• Targets conversation summarization and memory compression
• Safety instructions are progressively diluted across compression cycles
• Exploits abstractive summarization tendencies to rephrase or omit constraints
• Particularly dangerous in long-running agent sessions with periodic context compaction

The falsification of document metadata—including source attribution, publication date, authority scores, and credibility signals—to deceive an agent into trusting and prioritizing attacker-controlled information over legitimate sources.

• Forges citation chains and authority signals
• Exploits agents that weight sources by perceived credibility
• Combines with corpus poisoning to create seemingly authoritative fake documents
• Bypasses source-filtering defenses that rely on metadata integrity

A sophisticated technique that injects a malicious residual stream vector into the model's forward pass during inference. This directly overrides internal representations, forcing specific output behaviors without requiring any visible prompt manipulation.

• Operates at the representation engineering level
• Modifies attention head outputs or MLP activations
• Can bypass all input-level content filters
• Requires white-box access or model weight knowledge
• Related to steering vectors and concept erasure research