Sub-Question Output Contract and Variable Schema Prompt

This prompt is designed for platform engineers and RAG architects who need to decompose a complex user query into a set of independently retrievable sub-questions. It is intended to be wired directly into a retrieval pipeline where the output must be a strictly validated JSON object that downstream services can parse without ambiguity. The primary job-to-be-done is transforming a multi-part, comparative, or conditional user question into a machine-readable plan of attack, enabling a retrieval system to gather evidence for each facet of the query in parallel or in a defined sequence.

Use this prompt when a flat, single-vector similarity search is insufficient to answer the user's question. This typically occurs with queries that contain multiple clauses, require evidence from different data sources, or depend on conditional logic. The prompt enforces a strict JSON contract, making it suitable for high-reliability production systems where a malformed output will break the pipeline. It is not suitable for simple factoid lookups where a single retrieval round is sufficient, as the overhead of decomposition and validation adds unnecessary latency and complexity. The ideal user has a clear understanding of their available retrieval indices and can specify the expected evidence type for each sub-question.

Before implementing this prompt, ensure you have defined the target indices and evidence types your system supports. The prompt's [OUTPUT_SCHEMA] and [CONSTRAINTS] placeholders must be populated with your specific taxonomy. Do not use this prompt without a downstream validation function; the generated JSON must be verified for schema compliance, dependency integrity, and the absence of circular references before any retrieval calls are dispatched. If your use case involves high-stakes decisions, such as in finance or healthcare, a human review step should be inserted after decomposition to confirm the sub-questions fully and safely capture the user's intent before automated retrieval begins.

This template is designed for platform engineers integrating query decomposition into a RAG harness. It forces the model to output a predictable, machine-readable contract rather than free-text reasoning. The prompt uses explicit schema instructions, a JSON output constraint, and a dedicated field for dependency tracking to ensure that downstream orchestrators can execute sub-questions in the correct order or in parallel. Use this template when you need decomposition results that can be parsed, validated, and fed directly into a retrieval dispatch loop without manual cleanup.

textCopy
You are a query decomposition engine. Your only job is to break a complex user question into a set of smaller, independently answerable sub-questions.

## INPUT
User Question: [USER_QUESTION]
Available Retrieval Targets: [RETRIEVAL_TARGETS]
Conversation Context (if any): [CONVERSATION_CONTEXT]

## OUTPUT SCHEMA
Return a single JSON object with a key "sub_questions" containing an array of objects. Each object must have these fields:
- "id": string (unique identifier, e.g., "sq1", "sq2")
- "question": string (the self-contained sub-question, resolving all pronouns and implicit references)
- "depends_on": array of strings (list of sub-question IDs that must be answered before this one; empty array if independent)
- "target_index": string (must be one of the available retrieval targets listed above)
- "expected_evidence_type": string (one of: "fact", "list", "definition", "procedure", "comparison", "summary")
- "rationale": string (brief explanation of why this sub-question is necessary)

## CONSTRAINTS
1. Every sub-question must be fully self-contained. Resolve all pronouns, anaphora, and implicit context.
2. The union of all sub-questions must cover the full intent of the original question. Do not drop any part.
3. Sub-questions must be independently answerable. A single retrieval round against the specified target_index should be sufficient.
4. If the original question is already simple and cannot be decomposed, return a single sub-question.
5. Do not include the original question as a sub-question. Decompose it.
6. Order sub-questions so that independent ones come first, followed by dependent ones.
7. Do not include any text outside the JSON object.

## EXAMPLE
User Question: "Compare the pricing and security features of our enterprise plan with the pro plan, and tell me which one supports SAML."
Available Retrieval Targets: ["product_docs", "pricing_db", "security_whitepapers"]

Output:
{
  "sub_questions": [
    {
      "id": "sq1",
      "question": "What are the pricing details for the enterprise plan?",
      "depends_on": [],
      "target_index": "pricing_db",
      "expected_evidence_type": "fact",
      "rationale": "Extract enterprise pricing for comparison."
    },
    {
      "id": "sq2",
      "question": "What are the pricing details for the pro plan?",
      "depends_on": [],
      "target_index": "pricing_db",
      "expected_evidence_type": "fact",
      "rationale": "Extract pro pricing for comparison."
    },
    {
      "id": "sq3",
      "question": "What security features does the enterprise plan include?",
      "depends_on": [],
      "target_index": "security_whitepapers",
      "expected_evidence_type": "list",
      "rationale": "Extract enterprise security features for comparison."
    },
    {
      "id": "sq4",
      "question": "What security features does the pro plan include?",
      "depends_on": [],
      "target_index": "security_whitepapers",
      "expected_evidence_type": "list",
      "rationale": "Extract pro security features for comparison."
    },
    {
      "id": "sq5",
      "question": "Does the enterprise plan support SAML authentication?",
      "depends_on": [],
      "target_index": "product_docs",
      "expected_evidence_type": "fact",
      "rationale": "Answer the specific SAML question for enterprise."
    },
    {
      "id": "sq6",
      "question": "Does the pro plan support SAML authentication?",
      "depends_on": [],
      "target_index": "product_docs",
      "expected_evidence_type": "fact",
      "rationale": "Answer the specific SAML question for pro."
    }
  ]
}

To adapt this template, replace the placeholders with your application's runtime values. [USER_QUESTION] should be the raw user input. [RETRIEVAL_TARGETS] should be a JSON-serialized array of the index names or tool identifiers available in your system, such as ["vector_db_products", "sql_finance", "graph_entities"]. [CONVERSATION_CONTEXT] is optional but critical for multi-turn chat: include the last N turns so the model can resolve pronouns like "it" or "the second one." If your system uses a single retrieval backend, simplify the target_index constraint to a single value or remove the field. The expected_evidence_type field is a signal for downstream answer synthesis: a "fact" sub-question expects a short extractive answer, while a "list" sub-question expects multiple items. You can extend this enum to match your synthesis templates. After generating the JSON, always run it through a validation function that checks for missing IDs, circular dependencies, and coverage of the original question's intent. A broken dependency chain will cause your orchestrator to hang or skip questions, so validate before execution.

This prompt is designed to be a deterministic component inside a larger retrieval pipeline, not a standalone chat interface. The primary integration point is immediately after a complex user query is received and before any retrieval calls are made. The application should call the LLM with this prompt, parse the JSON output, and use the resulting array of sub-question objects to drive parallel or sequential retrieval requests. The strict JSON schema is the contract: if the model's output doesn't parse, the pipeline must not proceed with malformed queries.

Validation is mandatory before retrieval. Implement a JSON schema validator in your application code that checks every object in the sub_questions array for required fields (id, query_text, dependency_ids, target_index, expected_evidence_type). Reject any output where dependency_ids reference non-existent id values, where target_index contains unsupported index names, or where query_text is empty. If validation fails, retry the prompt once with the validation error message appended to the [CONSTRAINTS] section. After a second failure, log the raw output and the original user query for manual review, and fall back to a simpler single-query retrieval strategy. For high-stakes domains like healthcare or finance, route all validation failures to a human review queue instead of silently falling back.

Model choice and tool integration matter here. Use a model with strong JSON mode or structured output support (e.g., GPT-4o with response_format: { type: 'json_schema' }, Claude with tool-use for the output schema, or a fine-tuned open-weight model). Do not rely on prompt instructions alone for JSON compliance. Wire the output schema directly into the model's native structured output API when available. The target_index field should map to a controlled vocabulary of your actual retrieval backends—vector, keyword, graph, hybrid—and your retrieval dispatcher should use this field to route each sub-query to the correct index. For sub-questions with dependency_ids, queue them for execution only after their parent sub-question's retrieval results are available, and inject those results into the [CONTEXT] placeholder of the dependent query's retrieval call.

Observability and debugging require logging the full decomposition at each step. Record the original user query, the generated sub-questions, the validation result, and any retry attempts. In production, monitor the rate of validation failures, the distribution of target_index selections, and the average number of sub-questions generated. A sudden spike in sub-question count or a shift toward a single index type often signals a prompt drift or a change in user query patterns. Set up alerts for validation failure rates exceeding 5% and for any instance where the fallback single-query path is triggered, as these represent degraded retrieval quality that users will notice.