Zero-Width Character Detection and Removal Prompt

This prompt is for security engineers, platform engineers, and API gateway operators who need to detect and remove zero-width characters from model-generated text before that text reaches downstream systems. The primary job-to-be-done is preventing invisible data corruption, homograph attacks, and parser confusion caused by characters like U+200B (Zero Width Space), U+200C (Zero Width Non-Joiner), U+200D (Zero Width Joiner), U+FEFF (Byte Order Mark), and other Unicode codepoints that render as zero-width glyphs but carry semantic or structural meaning. The ideal user has already identified that model outputs are causing silent failures—strings that look identical but fail equality checks, URLs that break when copied, or identifiers that don't match database records—and needs a programmatic detection and removal step before the text enters logs, databases, or user-facing surfaces.

Use this prompt when you need an audit trail alongside the cleaned output. The prompt is designed to return both a sanitized string and a structured report of every character removed, including its Unicode codepoint, name, and original position. This is critical for security incident response, where you must prove what was removed and why, and for compliance workflows that require evidence of sanitization. The prompt assumes you are working with UTF-8 encoded text and that the model has access to Unicode character property knowledge. It is not suitable for binary data, non-text payloads, or situations where zero-width characters are intentionally meaningful—such as Unicode joiners in Arabic script or Devanagari, where removing U+200C or U+200D would corrupt legitimate text rendering. For those cases, use a context-aware approach that distinguishes semantic joiners from malicious or accidental insertions.

Do not use this prompt as your only defense against homograph attacks. Zero-width characters are one vector among many; internationalized domain name (IDN) homograph attacks, confusable character substitution, and bidirectional text spoofing require additional detection layers. This prompt is a post-generation sanitization step, not a complete security control. Before deploying, validate the output against your specific downstream parser requirements—some systems reject all non-ASCII characters, while others accept specific Unicode ranges. Pair this prompt with a harness that logs every removal decision, flags unexpected character categories for human review, and measures false-positive rates against a golden dataset of legitimate Unicode text that must survive sanitization intact.

This prompt template is designed for security and platform engineers who need to sanitize model outputs that contain zero-width characters. These invisible characters—such as zero-width spaces (U+200B), zero-width non-joiners (U+200C), zero-width joiners (U+200D), and word joiners (U+2060)—can cause data corruption, enable homograph attacks, or confuse downstream parsers. The prompt instructs the model to scan the input text, identify all zero-width characters, remove them, and produce a cleaned string alongside an audit report of every character removed, including its Unicode code point, name, and position.

textCopy
You are a text sanitization engine specialized in detecting and removing zero-width Unicode characters from input strings. Your task is to produce a cleaned output and a complete audit trail.

## INPUT
[INPUT]

## CONSTRAINTS
- Remove ALL zero-width characters, including but not limited to:
  - U+200B (Zero Width Space)
  - U+200C (Zero Width Non-Joiner)
  - U+200D (Zero Width Joiner)
  - U+FEFF (Zero Width No-Break Space / BOM)
  - U+2060 (Word Joiner)
  - U+2061–U+2064 (Invisible operators)
  - U+180E (Mongolian Vowel Separator)
  - U+00AD (Soft Hyphen)
- Do NOT remove visible characters, standard whitespace (space, tab, newline), or legitimate Unicode joiners that are part of valid grapheme clusters in [CONTEXT].
- Preserve the original meaning, casing, and visible structure of the text.
- If no zero-width characters are found, return an empty audit array and the original text unchanged.

## CONTEXT
[CONTEXT]

## OUTPUT_SCHEMA
Return a JSON object with exactly these fields:
{
  "cleaned_text": "string",
  "audit_report": [
    {
      "character": "string (the removed character itself, may appear empty)",
      "unicode_codepoint": "string (e.g., U+200B)",
      "unicode_name": "string (official Unicode name)",
      "position": "integer (0-based index in the original input)",
      "reason": "string (brief explanation of why it was removed)"
    }
  ],
  "total_removed": "integer",
  "original_length": "integer",
  "cleaned_length": "integer"
}

## EXAMPLES
[EXAMPLES]

## INSTRUCTIONS
1. Scan the entire [INPUT] string character by character.
2. Identify every zero-width character using the list in [CONSTRAINTS].
3. For each identified character, record its Unicode code point, official name, 0-based position in the original string, and a brief reason for removal.
4. Build the cleaned_text by removing all identified zero-width characters while preserving all other content exactly.
5. Populate the audit_report array in order of occurrence.
6. Set total_removed to the count of removed characters.
7. Set original_length and cleaned_length to the character counts before and after removal.
8. Return ONLY the JSON object. No markdown fences, no commentary.

To adapt this template, replace [INPUT] with the text you need to sanitize—this could be a model-generated response, a user-submitted string, or a concatenated payload from multiple sources. The [CONTEXT] placeholder should describe the expected content domain (e.g., 'English technical documentation', 'mixed-language chat messages', 'JSON payload for an API') so the model can distinguish legitimate Unicode joiners from malicious or accidental zero-width characters. The [EXAMPLES] placeholder should contain one or two few-shot examples showing both a contaminated input and the expected output JSON, especially demonstrating edge cases like zero-width characters at the start of a string, consecutive zero-width characters, and legitimate emoji sequences that use U+200D as a joiner. If you are processing high-risk content such as authentication tokens, security logs, or user-facing identifiers, add a [RISK_LEVEL] field set to 'high' and include an additional instruction requiring human review of the audit report before the cleaned text is used in production.

Integrating the zero-width character detection prompt into an application requires treating it as a post-generation sanitization step within your output pipeline. The prompt should be called immediately after receiving a model response and before that response is passed to any downstream parser, database, or user-facing surface. The harness must handle the prompt's structured output—a cleaned string and an audit report—and decide what to do with each. A common pattern is to place this prompt inside a middleware or interceptor function in your API gateway or model proxy, so every non-streaming response is automatically scanned. For streaming responses, buffer the full output before invoking the prompt, as zero-width characters can span chunk boundaries.

The implementation should include a validation layer that parses the prompt's JSON response and confirms the schema: a cleaned_string field and a removed_characters array with character, unicode_code_point, count, and positions for each finding. If the model fails to return valid JSON, implement a retry with a stricter schema instruction (e.g., 'You must respond with ONLY the JSON object and no other text'). After successful parsing, compare the length of the input string to the cleaned_string to detect discrepancies. Log every detection event with the full audit report, the model ID, the prompt version, and a timestamp. For high-security environments, quarantine any output where zero-width characters were found and route it for human review before it reaches users, especially if the characters could indicate a homograph attack or data exfiltration attempt.

Model choice matters. This task requires precise character-level reasoning, not creative generation. Use a model with strong instruction-following and JSON mode support, such as gpt-4o or claude-3.5-sonnet. Avoid smaller or older models that may hallucinate Unicode code points or miss zero-width joiners. Set temperature=0 to maximize deterministic behavior. The prompt should be called with a strict timeout (e.g., 2 seconds) since it operates on short strings; if it times out, fall back to a regex-based sanitizer as a safety net. The regex approach (/[\u200B-\u200D\uFEFF\u00AD\u2060]/g) is fast and reliable for known zero-width characters but will miss novel or obfuscated sequences, so treat it as a fallback, not a replacement. Wire both the prompt-based and regex-based results into your observability dashboard so you can track divergence between the two methods over time.

Avoid false-positive removal of legitimate Unicode joiners. Zero-width joiners (ZWJ, U+200D) are used correctly in emoji sequences (e.g., family emojis) and Indic scripts. Your harness should maintain a configurable allowlist of contexts where ZWJ is expected. Before flagging a ZWJ for removal, check whether it appears between known emoji codepoints or within script runs that require it. If your application domain is multilingual, invest in a pre-processing step that identifies the script of the surrounding text and adjusts the removal policy accordingly. For applications that process user-generated content in Arabic, Devanagari, or Sinhala, ZWJ removal can corrupt text rendering. The harness should log a warning instead of removing when ZWJ appears in these contexts, and escalate to a human reviewer if the count exceeds a threshold.

Finally, treat this prompt as a security control, not a formatting utility. Version it alongside your other security prompts, include it in your red-team test suite, and monitor its false-positive and false-negative rates in production. Add an eval harness that tests the prompt against a golden dataset of known malicious strings (e.g., homograph domain names, hidden text for prompt injection) and benign strings with legitimate Unicode (e.g., Arabic text with ZWJ, emoji sequences). Run these evals on every prompt change before deployment. If your application is in a regulated industry, ensure the audit report from this prompt is stored in an immutable log for compliance review.