Diacritic and Accent Normalization Prompt for Search Indexing

This prompt is for search engineers and platform developers who need to normalize model-generated text before it enters a search index. The core job is to strip or standardize diacritical marks (accents, umlauts, cedillas) so that queries for 'cafe' and 'café' return the same results. This is not a general-purpose text cleaning prompt; it is specifically designed for the indexing pipeline where token matching, not human readability, is the primary constraint.

Use this prompt when your search index treats accented and unaccented characters as distinct tokens, causing recall failures. It is appropriate when the model output is otherwise well-formed but contains inconsistent accent usage across languages. Do not use this prompt when accent marks carry semantic meaning that must be preserved (e.g., distinguishing Spanish 'si' from 'sí', or when the normalized text will be displayed to users without the original form available for reference). In those cases, store both the normalized and original forms.

The ideal user is a backend engineer or search relevance specialist who controls the ingestion pipeline and can wire this prompt into a pre-indexing transformation step. You need a clear language context for the input text; the normalization rules for French differ from those for German or Vietnamese. Before implementing, confirm that your downstream tokenizer and analyzer are also configured to handle the normalized output, or you will have solved one mismatch only to create another.

The following prompt template is designed to be copied directly into your prompt library or orchestration layer. It instructs the model to normalize text containing diacritics and accented characters according to a specified strategy, producing output that is consistent for search indexing pipelines. The placeholders allow you to inject the raw text, define the normalization rules, and specify the exact output format your downstream systems expect.

textCopy
You are a text normalization engine. Your task is to process the provided text and normalize all diacritic and accented characters according to a strict set of rules for search indexing.

### INPUT TEXT
[INPUT]

### NORMALIZATION RULES
Apply the following rules in order:
[NORMALIZATION_RULES]

### LANGUAGE CONTEXT
The primary language of the text is [LANGUAGE]. Apply language-specific normalization exceptions if any are defined in the rules.

### OUTPUT FORMAT
Return ONLY the normalized text. Do not include any explanations, notes, or markdown formatting. The output must be a single block of text.

### CONSTRAINTS
- Preserve all non-diacritic punctuation, whitespace, and line breaks exactly as they appear in the input.
- Do not alter the semantic meaning of any word.
- If a character has no defined normalization rule, leave it unchanged.

To adapt this template, replace the placeholders with concrete values. [INPUT] should be the raw, unnormalized text. [NORMALIZATION_RULES] must be a clear, ordered list of directives, such as 'Convert all characters to their ASCII equivalents using Unicode Normalization Form KD (NFKD)' or 'Strip accents from French characters but preserve the German 'ß' as 'ss'.' The [LANGUAGE] placeholder is critical for avoiding false-positive stripping in languages where diacritics are semantically significant. Before deploying, test this prompt with a golden dataset that includes edge cases like the Turkish 'İ/i' pair, Vietnamese tone marks, and combined characters to ensure your rules produce the expected output.

The diacritic normalization prompt is designed to sit between model-generated content and your search index. It should be called as a post-processing step after text generation but before tokenization and indexing. The prompt accepts raw text and returns normalized text with consistent accent handling. You'll need to decide whether normalization happens synchronously during ingestion or asynchronously in a batch repair job. For high-throughput search pipelines, consider batching multiple documents into a single prompt call to reduce API overhead, but keep batches small enough that a single failure doesn't block an entire batch.

Wire the prompt into your application with a validation layer that checks the output before indexing. At minimum, validate that the output length is within a reasonable tolerance of the input length (a sudden 50% shrinkage likely indicates a failure mode). Implement a language detection check on the input text and compare it against the normalization rules applied. For example, if the input is detected as German, the output must preserve 'ä', 'ö', 'ü' as distinct characters rather than stripping them to 'a', 'o', 'u'. Use a lookup table of language-specific normalization rules: French retains 'é', 'è', 'ê', 'ë' as distinct; Spanish normalizes 'ñ' carefully; Turkish distinguishes dotted and dotless 'i'. If the validator detects a mismatch between detected language and applied rules, flag the output for human review or route to a language-specific retry prompt with explicit instructions.

Implement a retry loop with a maximum of two attempts. On the first validation failure, append the specific validation error to the retry prompt as additional context: 'The previous normalization incorrectly stripped accents from [LANGUAGE] text. Preserve [SPECIFIC_CHARACTERS] as distinct characters.' If the second attempt also fails validation, log the failure with the input text, detected language, and both output versions for offline analysis. Do not silently index failed normalizations. Instead, index the original text with a metadata flag indicating normalization was attempted but failed, so search quality can be monitored. For model choice, prefer models with strong multilingual performance. Test across your top five languages by volume before deploying. Avoid using this prompt on text that has already been normalized, as double-normalization can strip intentional diacritics that carry semantic meaning in the target language.