Prompt Normalization

The core mechanism of prompt normalization is canonicalization—converting input into a single, unambiguous standard form. This process strips away encoding tricks attackers use to bypass filters.

• Resolves homoglyph attacks by mapping visually similar Unicode characters (e.g., Cyrillic 'а' vs. Latin 'a') to a canonical representation
• Decodes percent-encoding, HTML entities, and Unicode escape sequences into their raw form for consistent inspection
• Collapses zero-width characters (U+200B, U+FEFF) that attackers insert to break tokenization or keyword matching
• Normalizes whitespace, case, and character composition to eliminate trivial filter evasion

Normalization enforces a strict instructional hierarchy by restructuring prompts so that user data and system directives occupy distinct, non-overlapping regions. This prevents privilege escalation where user input is misinterpreted as a system command.

• Wraps user content in explicit delimiters (e.g., XML tags, triple backticks) that the model is trained to treat as data, not instructions
• Strips or escapes control sequences that mimic system-level syntax, such as fake delimiter characters or markdown that could confuse the parser
• Reorders prompt components to ensure system instructions always take precedence in the attention mechanism, regardless of user input length or content

Effective normalization must neutralize threats without destroying the semantic intent of legitimate user queries. The process distinguishes between malicious structural manipulation and benign content.

• Preserves the user's original meaning while removing only the syntactic attack surface
• Maintains multi-turn context integrity by normalizing each turn independently while preserving conversational coherence
• Avoids over-sanitization that would degrade model performance on legitimate, complex queries containing code snippets or structured data
• Applies context-aware parsing that understands when special characters are part of a genuine request (e.g., a user asking about SQL injection) versus an actual attack attempt

Prompt normalization functions as the first stage in a layered defense architecture, feeding sanitized output to downstream security controls for additional scrutiny.

• Feeds normalized text into adversarial prompt detection classifiers that analyze perplexity and intent
• Passes canonicalized input to a guard model for secondary policy evaluation before the primary model processes it
• Enables delimiter-based defenses to function reliably by ensuring user content cannot contain spoofed delimiter sequences
• Logs both raw and normalized prompts for auditability, enabling security teams to analyze blocked attacks and refine normalization rules

As models become multi-modal, normalization must extend beyond text to cover all input channels. Attackers can embed malicious instructions in image metadata, audio transcripts, or document structures.

• Extracts and normalizes text from images (OCR output, alt text, embedded metadata) before it reaches the model's context
• Processes audio transcriptions through the same canonicalization pipeline as direct text input
• Strips hidden layers, comments, and metadata from uploaded documents (PDFs, spreadsheets) that could contain injection payloads
• Applies consistent normalization rules across all modalities to prevent attackers from exploiting the weakest input channel

Normalization must operate with minimal latency to avoid degrading the user experience in real-time conversational applications. Efficient implementation is critical for production deployment.

• Executes as a streaming pre-processor that normalizes input incrementally as tokens arrive, rather than waiting for the complete prompt
• Uses compiled finite-state transducers or optimized regex engines for sub-millisecond canonicalization of common attack patterns
• Caches normalization results for repeated or similar inputs to reduce computational overhead in high-throughput systems
• Implements timeout boundaries to prevent resource exhaustion from adversarial inputs designed with pathological complexity (e.g., deeply nested encodings)

The foundational process of cleaning user-provided text to remove or neutralize potentially malicious control sequences before model processing. This includes stripping or escaping special characters, normalizing Unicode, and validating input structure.

• Key Actions: Character stripping, encoding normalization, length limiting
• Relationship: Prompt normalization is a sophisticated form of sanitization that goes beyond cleaning to restructure the prompt's semantic intent
• Limitation: Aggressive sanitization can break legitimate user inputs requiring special formatting

The process of converting input data into a single, standard, unambiguous representation to prevent attackers from bypassing filters using encoding tricks. This is a critical prerequisite to effective prompt normalization.

• Defense Against: Homoglyph attacks, zero-width character injection, double-encoding
• Example: Converting %27 (URL-encoded apostrophe) to ' before applying safety checks
• Failure Mode: Failing to canonicalize before validation allows encoded malicious payloads to slip through

A mitigation technique that uses special character sequences to clearly separate untrusted user input from trusted system instructions. This creates a structural boundary that makes it harder for user input to override system directives.

• Common Delimiters: XML tags, Markdown fences, or custom tokens like ---USER INPUT---
• Synergy: Prompt normalization often rewrites user input into a delimited block within a canonical template
• Weakness: Attackers may attempt to inject closing delimiters to break out of the designated input zone

A safety framework that prioritizes system-level instructions over user-level or tool-level instructions to prevent lower-privilege inputs from overriding core directives. This is a logical complement to structural normalization.

• Principle: System > User > Tool in authority ranking
• Implementation: Models are trained to recognize and respect privilege levels embedded in the prompt structure
• Normalization Role: Prompt normalization enforces this hierarchy by ensuring user input is always placed in a subordinate syntactic position

The use of classifiers, heuristics, or perplexity analysis to identify user inputs that are likely crafted to manipulate a model. This acts as a pre-filter before normalization occurs.

• Detection Methods: Perplexity scoring, semantic similarity to known attacks, trained binary classifiers
• Integration: Flagged prompts can be blocked outright or routed to a stricter normalization pipeline
• Trade-off: High false-positive rates can degrade user experience for legitimate complex queries

A defensive technique that strictly segregates different information sources within a prompt to prevent cross-contamination and privilege escalation. This is the architectural principle that prompt normalization operationalizes.

• Goal: Prevent user input from being interpreted as system instructions
• Techniques: Explicit context markers, attention masking, separate embedding spaces
• Normalization Connection: Rewriting user input into a standardized, clearly bounded format is the primary enforcement mechanism