Cross-Lingual Query Decomposition Prompt Template

This prompt is designed for RAG architects and search engineers who must handle complex, multi-part user questions in multilingual retrieval systems. The core job-to-be-done is transforming a single compound question—posed in one language—into a set of atomic, independently retrievable sub-questions, each translated into the target index languages. This is necessary when a user asks a question that contains comparisons, conditional logic, temporal sequences, or requires evidence from multiple documents that must be assembled stepwise. A single translated query will fail in these scenarios because the evidence is scattered across documents in different languages, and the retrieval system needs to execute multiple hops in a specific order to gather all the pieces.

Use this prompt when the source question exhibits clear multi-hop structure: 'Compare the warranty terms for Product X in the EU and the US, then tell me which is longer' requires retrieving two separate documents, likely in different languages, before a comparison can be made. Similarly, 'What were the side effects reported in the German trial, and did the Japanese study find the same?' demands sequential retrieval where the second query depends on the findings of the first. The prompt outputs a dependency graph that your retrieval orchestrator can parse to schedule parallel and sequential retrieval calls correctly. Do not use this prompt for simple factual lookups that a single translated query can satisfy, such as 'What is the capital of France?' or 'When was Company Y founded?'—these only add latency and token cost without improving recall. Also avoid it when the question's complexity is purely linguistic rather than evidentiary; if a question is verbose but ultimately asks for a single fact, a direct translation prompt is more efficient.

Before integrating this prompt into your pipeline, verify that your retrieval backend supports multi-hop orchestration. The output dependency graph assumes you have a system capable of executing retrieval steps in order, passing intermediate results as context to subsequent steps. If your current RAG setup is a single-round retriever-reader loop, you will need to build or adopt an agentic retrieval layer first. Start by testing the prompt against a golden set of 20-30 known multi-hop, cross-lingual questions from your domain, and manually verify that the decomposed sub-questions are both necessary and sufficient to answer the original query. A common failure mode is over-decomposition, where the model breaks a question into too many granular steps, increasing latency without improving answer quality. Set a maximum sub-question limit (e.g., 5) in your orchestration logic and log cases where the limit is hit for later review.

This template is designed for RAG architects who need to handle complex questions in multilingual settings. It takes a user's query in a source language, breaks it down into atomic sub-questions, translates each into the target retrieval languages, and defines the order in which they must be answered. The output is a structured JSON object that can be directly consumed by an orchestration layer to execute parallel and sequential retrieval steps.

codeCopy
You are a query decomposition engine for a multilingual RAG system. Your job is to analyze a complex user query and break it into a set of independent sub-questions that can be answered separately. Each sub-question must be translated into the target index languages. You must also define a dependency graph showing which sub-questions depend on the answers of others.

## INPUT
User Query: [USER_QUERY]
Source Language: [SOURCE_LANGUAGE]
Target Index Languages: [TARGET_LANGUAGES]

## CONSTRAINTS
- Decompose the query into the smallest number of atomic sub-questions that fully cover the user's intent.
- Each sub-question must be self-contained and answerable without referring to other sub-questions.
- Translate every sub-question into each of the target index languages.
- Identify dependencies: if answering sub-question B requires the answer to sub-question A, mark A as a prerequisite of B.
- If the query is already simple and cannot be decomposed, return a single sub-question with no dependencies.
- Do not answer the questions. Only generate the decomposed and translated queries.

## OUTPUT SCHEMA
Return a single JSON object with the following structure:
{
  "original_query": "string",
  "source_language": "string",
  "target_languages": ["string"],
  "sub_questions": [
    {
      "id": "string (e.g., 'q1')",
      "description": "string (the sub-question in the source language)",
      "translations": {
        "[language_code]": "string (translated sub-question)"
      },
      "depends_on": ["string (list of sub-question IDs that must be answered first)"]
    }
  ]
}

## EXAMPLE
User Query: "What were the revenue impacts of the new privacy regulations in Germany and how did our competitor respond?"
Source Language: "en"
Target Index Languages: ["en", "de"]

Output:
{
  "original_query": "What were the revenue impacts of the new privacy regulations in Germany and how did our competitor respond?",
  "source_language": "en",
  "target_languages": ["en", "de"],
  "sub_questions": [
    {
      "id": "q1",
      "description": "What are the new privacy regulations in Germany?",
      "translations": {
        "en": "What are the new privacy regulations in Germany?",
        "de": "Was sind die neuen Datenschutzbestimmungen in Deutschland?"
      },
      "depends_on": []
    },
    {
      "id": "q2",
      "description": "What were the revenue impacts of these regulations on our company?",
      "translations": {
        "en": "What were the revenue impacts of these regulations on our company?",
        "de": "Welche Umsatzauswirkungen hatten diese Vorschriften auf unser Unternehmen?"
      },
      "depends_on": ["q1"]
    },
    {
      "id": "q3",
      "description": "How did our competitor respond to these regulations?",
      "translations": {
        "en": "How did our competitor respond to these regulations?",
        "de": "Wie hat unser Wettbewerber auf diese Vorschriften reagiert?"
      },
      "depends_on": ["q1"]
    }
  ]
}

To adapt this template, replace the placeholders with your application's runtime values. The [USER_QUERY] should be the raw, unmodified input from the user. [SOURCE_LANGUAGE] should be an ISO 639-1 code detected upstream. [TARGET_LANGUAGES] is a list of language codes for the indexes you intend to query. The output schema is designed to feed directly into an orchestration engine: execute all sub-questions with empty depends_on arrays in parallel, then use their results as context when executing dependent sub-questions. For high-risk domains such as legal or financial analysis, add a [RISK_LEVEL] placeholder and include an instruction to flag sub-questions that require human review before retrieval.

Before deploying, validate the output against the schema. Common failure modes include the model answering the questions instead of decomposing them, generating sub-questions that are not truly atomic, or producing translations that drift semantically from the source. Implement a post-processing validator that checks for the presence of all required fields, verifies that dependency IDs reference existing sub-questions, and confirms that translations exist for every target language. For production systems, log the decomposition graph alongside retrieval results to trace multi-hop answer assembly and diagnose failures.

This prompt is not a standalone translator; it is a planning step in a multi-hop retrieval pipeline. The application must receive the decomposed sub-questions and their dependency graph, then execute retrieval for each sub-question in the specified target language against the appropriate index. The harness should treat the model output as a structured execution plan, not a final answer. Because the prompt generates a JSON schema with a dependency_graph, the application layer must parse this graph, resolve dependencies, and sequence retrieval calls so that sub-questions depending on prior results wait for those results before execution. A common mistake is to fire all sub-questions in parallel without respecting the dependency order, which breaks multi-hop reasoning chains.

Validation and Schema Enforcement: Before any retrieval call, validate the model's JSON output against the expected schema. Check that every sub-question has a non-empty text field, a valid ISO 639-1 target_language code, and that all depends_on indices reference existing sub-question IDs. Reject outputs where a dependency references a non-existent sub-question or creates a cycle. Use a JSON Schema validator in your application code—do not rely on the model to self-correct without a validation gate. If validation fails, retry the prompt once with the validation error message appended to the [CONSTRAINTS] field. After two consecutive failures, log the raw output and escalate to a human reviewer or a fallback single-query translation prompt.

Retrieval Execution and Context Assembly: For each validated sub-question, execute retrieval against the index corresponding to its target_language. Store the retrieved chunks keyed by sub-question ID. When a sub-question depends on a prior result, inject the retrieved context from the dependency into the retrieval query for the dependent sub-question—either by appending it as a [CONTEXT] block or by using it to refine the query vector. After all sub-questions are resolved, assemble the full evidence map (sub-question ID → retrieved chunks) and pass it to a downstream answer-generation prompt. Log the decomposition plan, retrieval latency per sub-question, and any dependency-resolution failures for observability.

Model Choice and Latency Budget: This prompt benefits from models with strong instruction-following and JSON output discipline, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Expect higher latency than a single translation prompt because the model must reason over the query structure and generate a plan. Set a timeout of 10–15 seconds for the decomposition call. If the source query is short and clearly single-hop, skip decomposition entirely and route directly to a simpler cross-lingual translation prompt. Use a lightweight classifier or keyword heuristic (e.g., presence of 'and', 'then', 'after', or multiple question marks) to decide whether decomposition is warranted before invoking this more expensive prompt.

Failure Modes and Guardrails: The most common failure is the model generating sub-questions that are not truly independent, causing redundant retrieval or circular dependencies. Monitor for sub-questions that rephrase the same intent in slightly different words—deduplicate them by cosine similarity before retrieval. Another failure is language drift, where a sub-question is generated in the wrong target language. Validate each target_language code against your configured index languages and reject sub-questions targeting unsupported languages. Finally, if the dependency graph is too deep (more than 3 hops), consider truncating or flagging for human review, as retrieval errors compound across hops and degrade the final answer quality.