Canonicalization

The canonicalization function must produce the same output regardless of how many times it is applied. Applying the function once must yield the same result as applying it ten times.

• Property: f(x) = f(f(x))
• Why it matters: Prevents attackers from using recursive encoding (e.g., double URL-encoding) to bypass a single-pass filter.
• Example: If %2527 decodes to %27, a second pass decodes it to '. An idempotent function resolves it to ' in a single, atomic operation.

The process must preserve the semantic meaning of the input while reducing it to its simplest form. No critical data should be destroyed during normalization.

• Key distinction: Converting ../ to its canonical path is lossless for path resolution, but stripping it entirely is lossy and dangerous.
• Example: Unicode normalization forms like NFC vs. NFD. Choosing the wrong form can break string comparisons, but the chosen form must consistently represent the same abstract characters.

The system must define and enforce one true representation for every data type. There can be no ambiguity about which form is the standard.

• File paths: A single canonical form (e.g., absolute path with no trailing slashes or dot segments).
• Unicode: A single normalization form (NFC or NFKC) enforced at the boundary.
• JSON keys: A strict ordering and whitespace policy.
• Why it matters: If admin and ADMIN are treated as distinct, an attacker can register ADMIN to impersonate the admin user.

Canonicalization must occur at the trust boundary—the exact point where data crosses from an untrusted zone to a trusted one. It is not a step to be deferred.

• Rule: Convert to canonical form before validation, never after.
• Anti-pattern: Validating a filename for path traversal characters, then later decoding it for storage.
• Example: A web application firewall must canonicalize a request's URL and body before applying its rule set, otherwise an encoded attack like %2e%2e%2f passes through undetected.

The algorithm must fully resolve all layers of encoding and nested representations until a raw, atomic form is reached. Partial resolution is a security vulnerability.

• Process: Decode URL-encoding -> Decode HTML entities -> Normalize Unicode -> Resolve path traversal.
• Attack vector: An attacker sends %253Cscript%253E. A single decode pass yields %3Cscript%3E, which is still benign to a naive filter. A second pass yields <script>. Recursive resolution catches this on the first pass.

Given the same input byte stream, the canonicalization function must always produce the exact same output, on every platform, in every environment.

• No system call variance: Relying on the local file system's realpath() can yield different results on different OS kernels.
• No locale dependence: String case-folding must use a locale-independent method (e.g., toUpperCase(Locale.ROOT) in Java) to avoid the 'Turkish I' problem.
• Why it matters: A distributed system where different nodes canonicalize the same input differently creates a security gap that an attacker can exploit by targeting a specific node.

The process of cleaning and normalizing user-provided text to remove or neutralize potentially malicious control sequences before model processing. This is the direct implementation layer where canonicalization logic is applied.

• Strips or encodes dangerous characters
• Normalizes Unicode to a single form (e.g., NFC)
• Validates against an allowed schema
• Prevents homoglyph attacks and zero-width character injection

The process of rewriting or restructuring user prompts into a safe, canonical form before they are combined with system instructions. This goes beyond character-level canonicalization to address semantic structure.

• Reorders clauses to a standard template
• Removes redundant or obfuscating language
• Resolves multi-turn injection attempts
• Ensures delimiter-based defense boundaries remain intact

A technique that uses visually similar characters from different alphabets to bypass text-based filters while appearing normal to a human reviewer. Canonicalization is the primary defense.

• Replaces Latin 'a' with Cyrillic 'а' (U+0430)
• Evades keyword blocklists and pattern matching
• Mitigated by Unicode normalization (NFKC)
• Often combined with prompt injection payloads

An attack that inserts invisible Unicode characters into text to break tokenization or keyword filtering without being visible to the user. Canonicalization strips these characters before processing.

• Uses characters like U+200B (Zero-Width Space)
• Disrupts tokenizer boundaries
• Hides malicious strings from human review
• Defeated by stripping control characters during input sanitization

A mitigation technique that uses special character sequences to clearly separate untrusted user input from trusted system instructions. Canonicalization ensures these delimiters cannot be spoofed or escaped.

• Uses markers like --- USER INPUT ---
• Prevents prompt injection via boundary confusion
• Requires canonicalization to normalize delimiter-like user text
• Works in tandem with instructional hierarchy frameworks

A safety framework that prioritizes system-level instructions over user-level or tool-level instructions to prevent lower-privilege inputs from overriding core directives. Canonicalization ensures privilege boundaries are unambiguous.

• System > User > Tool priority ordering
• Prevents privilege escalation via injection
• Relies on canonical input representation
• Core component of system prompt hardening strategies