Language Detection for RAG Pipeline Routing Prompt

This prompt is designed for AI platform engineers and MLOps teams who need to route user queries to language-specific vector indexes within a Retrieval-Augmented Generation (RAG) system. The core job-to-be-done is deterministic language identification at the ingress layer of a multilingual RAG pipeline, where sending a query to the wrong index—for example, routing a German question to an English knowledge base—directly degrades answer quality, breaks citation grounding, and erodes user trust. The ideal user is an engineer building a production retrieval system who needs a structured, testable language detection step that outputs a standardized language code, a confidence score, and an explicit index routing decision before the retrieval call is made.

Use this prompt when your RAG pipeline serves multiple language-specific knowledge bases or embedding collections and you need a reliable, auditable routing decision. It is appropriate for queries of moderate length—typically 10 to 500 words—where sufficient lexical and syntactic signal exists for disambiguation. The prompt expects a raw user query string as input and produces a JSON object containing the detected language, a confidence score between 0 and 1, and a target index identifier. It includes explicit handling for ambiguous or low-confidence cases by returning a fallback index and an ambiguity flag, preventing silent misrouting. The prompt is designed to be wired into a pre-retrieval middleware layer, where its output can be logged, validated, and used to select the correct vector store and embedding model before any search is executed.

Do not use this prompt for real-time streaming audio transcription, for inputs under 5 words where language signal is insufficient, or for documents containing multiple languages in equal proportion. It is not a translation prompt—it detects and routes, but does not transform the query. For code-switched inputs where two languages are interleaved within a single sentence, use the Mixed-Language Input Detection Prompt instead. For CJK disambiguation where character overlap between Chinese, Japanese, and Korean creates ambiguity, pair this prompt with the CJK Language Disambiguation Prompt. If your pipeline handles regulated content where language implies jurisdictional data residency requirements, combine this prompt with the Language Detection for Sensitive Data Handling Policy Prompt to ensure compliance-aware routing. Always validate the output against a golden dataset of known-language queries before deployment, and monitor production confidence score distributions to detect drift in your user population's language mix.

This prompt template is designed to sit at the ingress point of a multilingual retrieval-augmented generation (RAG) system. Its job is to accept a user query, detect the primary language, and produce a structured routing decision that downstream orchestration code can act on. The template uses square-bracket placeholders for all variable inputs, constraints, and configuration so you can adapt it to your specific language indexes, supported locales, and routing logic without rewriting the core instruction.

textCopy
You are a language detection router for a multilingual RAG system. Your task is to analyze the user's query, determine its primary language, and output a structured routing decision.

[INPUT]
User Query: "[USER_QUERY]"

[CONTEXT]
Supported Languages and Index Mappings:
[LANGUAGE_INDEX_MAPPINGS]

Default Fallback Language: [DEFAULT_FALLBACK_LANGUAGE]
Default Fallback Index: [DEFAULT_FALLBACK_INDEX]

[CONSTRAINTS]
1. Detect the primary language of the query. If the query contains multiple languages, identify the dominant one.
2. If the query contains foreign-language named entities (e.g., a French company name in an English query), do not let them override the primary language detection.
3. If the query contains technical terms borrowed from another language, treat them as part of the primary language's lexicon.
4. If the detected language is not in the Supported Languages list, route to the Default Fallback.
5. If the query is too short to disambiguate (fewer than [MINIMUM_CONFIDENCE_LENGTH] words), set confidence to "low" and flag for clarification.
6. Do not translate the query. Preserve the original text for downstream retrieval.

[OUTPUT_SCHEMA]
Return a single JSON object with the following fields:
{
  "detected_language": "ISO 639-1 code or 'unknown'",
  "confidence": "high | medium | low",
  "routing_target": "index identifier from the mapping or fallback",
  "requires_clarification": true | false,
  "clarification_reason": "explanation if clarification is needed, otherwise null",
  "original_query": "the exact user query, unmodified"
}

[EXAMPLES]
Example 1:
User Query: "What is the capital of France?"
Output: {"detected_language": "en", "confidence": "high", "routing_target": "en-index-v2", "requires_clarification": false, "clarification_reason": null, "original_query": "What is the capital of France?"}

Example 2:
User Query: "Was ist die Hauptstadt von Frankreich?"
Output: {"detected_language": "de", "confidence": "high", "routing_target": "de-index-v1", "requires_clarification": false, "clarification_reason": null, "original_query": "Was ist die Hauptstadt von Frankreich?"}

Example 3:
User Query: "Tell me about the restaurant 'La Maison d'Or' in Paris"
Output: {"detected_language": "en", "confidence": "high", "routing_target": "en-index-v2", "requires_clarification": false, "clarification_reason": null, "original_query": "Tell me about the restaurant 'La Maison d'Or' in Paris"}

Example 4:
User Query: "config"
Output: {"detected_language": "unknown", "confidence": "low", "routing_target": "en-index-v2", "requires_clarification": true, "clarification_reason": "Query too short to disambiguate language. Only 1 word provided.", "original_query": "config"}

[RISK_LEVEL]
This routing decision controls which knowledge base and embedding model are used. Incorrect routing may return irrelevant or untranslated results. If confidence is "low" and the query is short, prefer clarification over silent misrouting.

To adapt this template, replace the placeholders with your production values. [LANGUAGE_INDEX_MAPPINGS] should be a structured list mapping each supported ISO 639-1 code to its corresponding vector index identifier and embedding model name. [MINIMUM_CONFIDENCE_LENGTH] is a tunable threshold—start with 3 words and adjust based on your query patterns. The examples section is critical for teaching the model how to handle named entities and short queries; replace them with examples drawn from your actual query logs. Before deploying, run this prompt against a golden dataset of at least 200 queries covering all supported languages, mixed-language inputs, short queries, and edge cases with foreign named entities. Measure routing accuracy, clarification rate, and latency. If your system cannot tolerate any misrouting, add a human review step for all confidence: "low" outputs before the query reaches retrieval.

The language detection prompt is not a standalone utility; it is a critical routing component in a retrieval-augmented generation pipeline. When a user query arrives, the prompt must execute before any embedding lookup or document retrieval occurs. The detected language code determines which vector index to query, which embedding model to use, and which response generation locale to activate. A misclassification here silently breaks downstream retrieval quality—the system retrieves documents in the wrong language, generates responses the user cannot read, or wastes compute on indexes that contain no relevant content. Wire this prompt as a synchronous pre-retrieval step with a strict latency budget: 200–500ms is a reasonable target for most production systems, given that the prompt processes only the raw query text and returns a structured language label.

The implementation should follow a validate-then-route pattern. After the model returns a JSON payload containing the language code and confidence score, run a server-side validator that checks: (1) the language code exists in your supported language list, (2) the confidence score exceeds your configured threshold (start at 0.85 and calibrate downward based on eval results), and (3) the output schema matches the expected fields exactly. If validation fails, retry once with the same prompt—transient model failures or malformed JSON are the most common causes. If the retry also fails or returns confidence below threshold, route the query to a multilingual fallback index that contains documents across all supported languages. This fallback path prevents dropped queries and gives you a safety net while you tune the detection prompt. Log every routing decision—detected language, confidence score, retry count, and final index selected—so you can measure misrouting rates and identify language pairs where detection accuracy degrades.

For model selection, prefer a fast, cost-efficient model for this classification task. A lightweight model like GPT-4o-mini, Claude Haiku, or a fine-tuned open-weight classifier will handle language detection reliably without adding significant latency or cost to every query. Avoid routing this prompt to a large reasoning model—language detection is a pattern-matching task that does not benefit from chain-of-thought or extended context windows. If you are processing high query volumes, consider batching language detection requests where your architecture permits, or deploy a dedicated classification endpoint with connection pooling to reduce per-request overhead. For RAG systems that support code-switched or mixed-language queries, extend the validator to handle multi-language outputs and route to multiple indexes when the prompt returns more than one language above the confidence threshold.

The most dangerous failure mode in production is silent misrouting: the prompt returns a plausible but incorrect language code with high confidence, and the query flows to the wrong index without any alert. To catch this, implement an eval harness that runs alongside your production pipeline. Sample a percentage of routed queries (start with 5%), run them through a human-reviewed language verification step or a secondary detection model, and compare the results against the primary prompt's output. Set an alert threshold—if the disagreement rate exceeds 2% for any language pair, trigger a review of that pair's detection performance. For high-risk deployments where retrieval quality directly impacts user trust or regulatory compliance, add a human-in-the-loop confirmation step for queries where the confidence score falls between 0.70 and 0.85, or where the detected language is a known difficult case (e.g., short queries, CJK disambiguation, transliterated text). This adds latency but prevents the worst-class misrouting errors from reaching users.