Transliterated Text Language Detection Prompt

This prompt is designed for platform engineers and AI infrastructure teams building multilingual routing middleware. Its core job is to detect the source language of text that has been romanized or transliterated into Latin script—inputs like Franco-Arabic chat messages, Pinyin without tone marks, romanized Hindi typed on a QWERTY keyboard, or romanized Russian. Standard language detection libraries and APIs typically fail on these inputs because they rely on native-script statistical signals (e.g., Cyrillic characters, Devanagari conjuncts) that are absent after transliteration. This prompt fills that gap by reasoning about romanization patterns, common transliteration schemes, and lexical clues to produce a BCP 47 language code and the detected transliteration scheme.

Use this prompt when your ingestion pipeline receives user-generated text that 'sounds like' one language but is typed in another script. It is particularly valuable for chat platforms, customer support queues, and content moderation systems where users romanize their native language for convenience or keyboard limitations. The prompt expects inputs of at least 10 characters—shorter strings rarely carry enough signal to disambiguate romanization schemes. It also assumes the romanization follows a recognizable pattern (e.g., ISO 233 for Arabic, Hanyu Pinyin for Mandarin, Hunterian for Hindi) rather than being ad-hoc or deliberately obfuscated. When the transliteration scheme is inconsistent, the prompt will flag ambiguity rather than guess.

Do not use this prompt for native-script text—standard language detectors handle those inputs more efficiently and at lower cost. Avoid it for inputs under 10 characters, single-word queries, or text where the romanization is intentionally scrambled to evade detection. If your pipeline already has a native-script language detector, wire this prompt as a fallback that activates only when the primary detector returns low confidence on Latin-script input. For mixed inputs containing both native script and romanized segments, pre-segment the text and route each segment separately. The next step after reading this section is to review the prompt template and adapt its placeholders to your specific routing taxonomy and supported language set.

This prompt template is designed to identify the original language of text that has been written in a non-native script, most commonly Latin characters. It targets scenarios like Arabic chat alphabet (Arabizi), Pinyin without tone marks, romanized Hindi, or romanized Japanese. The prompt instructs the model to analyze the input, determine the most likely source language, and identify the transliteration scheme when detectable. Use this as a starting point for building language-aware routing middleware, multilingual search systems, or content moderation pipelines where script-based detection alone is insufficient.

textCopy
You are a specialized language detection system. Your task is to analyze text that may be transliterated or romanized and identify the original source language.

INPUT:
[INPUT]

INSTRUCTIONS:
1. Analyze the provided text to determine if it is written in a non-native script, specifically Latin characters representing another language.
2. Identify the most likely source language (e.g., Arabic, Mandarin Chinese, Hindi, Russian, Japanese).
3. If a specific transliteration scheme is detectable (e.g., Arabizi/Franco-Arabic, Pinyin, ITRANS, Hepburn Romaji), name it.
4. If the text is already in its native script or is natively English, state that no transliteration is detected.
5. Provide a confidence score (0.0 to 1.0) for your primary language identification.
6. If multiple source languages are plausible, list them with individual confidence scores and flag the result as ambiguous.

OUTPUT_SCHEMA:
[OUTPUT_SCHEMA]

CONSTRAINTS:
[CONSTRAINTS]

To adapt this template, replace the bracketed placeholders with concrete values for your application. The [INPUT] placeholder should receive the raw text string. The [OUTPUT_SCHEMA] is critical for integration; replace it with a strict JSON schema, such as {"detected_language": "string", "language_code": "string (ISO 639-1)", "transliteration_scheme": "string | null", "confidence_score": "number", "is_ambiguous": "boolean", "alternative_languages": [{"language": "string", "code": "string", "score": "number"}]}. The [CONSTRAINTS] placeholder should be replaced with operational rules, for example: "If confidence is below 0.7, set is_ambiguous to true and do not route automatically. Do not guess the language for inputs under 5 characters." For high-stakes routing, always pair this prompt with a validation layer that checks the output against the expected schema and logs low-confidence results for human review.

This prompt is designed to sit inside a classification middleware layer that receives romanized or transliterated text before it reaches downstream language-specific models, translation services, or routing logic. The harness must treat the prompt output as a structured signal—not a final answer—and validate it against expected schemas, confidence thresholds, and business rules before acting on it. Because transliterated text is inherently ambiguous (multiple source languages can produce the same romanized string), the implementation must handle multi-hypothesis outputs and low-confidence results gracefully.

Wire the prompt into your application as a pre-processing step with a strict JSON output contract. Define the expected output schema as { "source_language": "string (ISO 639-1 code)", "transliteration_scheme": "string | null", "confidence": "float (0-1)", "alternative_languages": ["string"], "notes": "string | null" }. On the application side, validate that source_language is a valid ISO 639-1 code, confidence is a float between 0 and 1, and alternative_languages is an array of valid codes. If validation fails, retry once with a stricter prompt that includes the validation error message. If the retry also fails, log the raw output and route to a human review queue or a fallback und (undetermined) language path. For high-throughput systems, batch inputs and process them asynchronously, but keep a synchronous path for latency-sensitive use cases under 200ms.

Model choice matters here. Use a model with strong multilingual and transliteration awareness—Claude 3.5 Sonnet, GPT-4o, or Gemini 1.5 Pro are good starting points. Smaller or older models frequently confuse romanization schemes (e.g., treating Pinyin as Vietnamese quốc ngữ or Arabic chat alphabet as Malay). Implement a confidence threshold gate: if confidence is below 0.7, route to a clarification prompt that asks the user to specify their intended language, or fall back to a multi-language processing pipeline that runs detection across all plausible candidates. Log every detection result with the input text hash, detected language, confidence, and downstream routing decision for later eval analysis and threshold tuning.

For eval, build a test harness with known transliterated examples: Arabic chat alphabet ('kayf halak'), Pinyin without tones ('ni hao'), romanized Hindi ('aap kaise hain'), and ambiguous cases like 'merci' (French or romanized Arabic?). Measure precision and recall per language pair, and track the false-positive rate where ambiguous romanization is assigned to the wrong language. Set up a periodic eval run against your golden dataset whenever the prompt or model version changes. If this prompt feeds into a translation layer, measure end-to-end translation quality degradation when language detection is wrong—this is the real business metric that justifies investment in detection accuracy.

Do not use this prompt as the sole routing mechanism for high-stakes workflows (healthcare intake, legal document processing, financial transactions) without human review on low-confidence outputs. The ambiguity of transliterated text means even the best models will make errors on short inputs, proper names, and cross-lingual homographs. Pair this prompt with a user-facing language confirmation step when confidence is below your threshold, and always provide an escape path for users to manually select their language.