Retrieval Backend Capability Analysis Prompt for Query Planning

This prompt is for architects and infrastructure engineers designing heterogeneous retrieval systems who need to map a user's query requirements to the most capable backend configuration. The job-to-be-done is not to rewrite a query, but to analyze the capabilities of available backends—dense vectors, sparse keyword indexes, graph traversals—and produce a reasoned plan for which backend or combination of backends should serve a given query. Use this when you have multiple live indexes with different strengths and you need an automated dispatch layer that understands each backend's schema, performance profile, and constraints before a single retrieval call is made.

The ideal user is a technical decision maker or platform engineer who already has a catalog of backend capabilities (e.g., 'dense index supports semantic similarity but not exact term matching', 'graph index supports 2-hop traversals on entity relationships', 'sparse index supports BM25 with field-level boosting'). You must supply this capability catalog as part of the [BACKEND_CAPABILITIES] input. The prompt is not a substitute for a vector database or a search engine; it is a reasoning layer that sits before retrieval to prevent wasted compute and bad results from sending queries to the wrong index. Do not use this prompt when you have a single backend, when you lack a formal capability specification, or when the query type is already statically routed by a rule-based classifier.

This prompt is high-risk in production because a capability mismatch can silently degrade retrieval quality. A query that requires exact entity matching sent to a dense embedding index will return semantically related but factually wrong documents. Always pair this prompt's output with validation checks: confirm that the recommended backend actually exists in your infrastructure, that its schema supports the required query shape, and that the capability claims in the plan match the backend's documented features. For regulated or high-stakes domains, require human review of the capability mapping before the plan is executed, and log every dispatch decision with the rationale for auditability.

This prompt template is designed for architects and infrastructure engineers who need to systematically evaluate available retrieval backends—dense vectors, sparse keywords, graph traversals, and hybrid indexes—against the specific requirements of a user query. Rather than guessing which backend to use, this prompt forces the model to reason about each backend's strengths, limitations, and schema constraints before producing a concrete query plan. The output is a structured capability-to-requirement mapping that can be consumed by an orchestration layer or reviewed by a human before execution.

textCopy
You are a retrieval architecture analyst. Your job is to evaluate available retrieval backends against a user query and produce a query plan that maps requirements to the most capable backend configuration.

## AVAILABLE BACKENDS
[BACKEND_CATALOG]

## USER QUERY
[USER_QUERY]

## QUERY CONTEXT
- User role: [USER_ROLE]
- Session history summary: [SESSION_CONTEXT]
- Known constraints: [CONSTRAINTS]

## OUTPUT SCHEMA
Return a JSON object with this structure:
{
  "query_analysis": {
    "intent": "string",
    "required_capabilities": ["semantic_similarity", "exact_term_match", "graph_traversal", "temporal_range", "metadata_filtering"],
    "complexity": "simple | compound | multi_hop",
    "precision_recall_profile": "high_precision | high_recall | balanced"
  },
  "backend_assessments": [
    {
      "backend_id": "string",
      "backend_type": "dense_vector | sparse_keyword | graph | hybrid",
      "capability_match_score": 0.0-1.0,
      "matched_capabilities": ["string"],
      "missing_capabilities": ["string"],
      "schema_constraints": ["string"],
      "recommended": true | false,
      "rationale": "string"
    }
  ],
  "recommended_plan": {
    "primary_backend": "backend_id",
    "fallback_backends": ["backend_id"],
    "parallel_backends": ["backend_id"],
    "execution_order": ["backend_id"],
    "query_variants": {
      "backend_id": "reformulated query string"
    },
    "fusion_strategy": "reciprocal_rank | weighted_score | round_robin | none",
    "confidence": 0.0-1.0,
    "risks": ["string"]
  }
}

## ANALYSIS RULES
1. Evaluate every backend in the catalog against the query's required capabilities.
2. Do not recommend a backend that lacks a required capability unless no backend satisfies it.
3. Flag schema mismatches explicitly—if a backend requires metadata filters the query cannot provide, mark it as a constraint violation.
4. For multi-hop queries, specify execution order and dependency between backends.
5. If confidence is below 0.7, include a human_review_required flag and explain what information is missing.
6. Prefer parallel execution when backends are independent; prefer sequential when one backend's output feeds another.

## CONSTRAINTS
[CONSTRAINTS]

## EXAMPLES
[EXAMPLES]

To adapt this template, replace [BACKEND_CATALOG] with a structured description of each available backend, including its type, schema, supported query operations, and known limitations. The [USER_QUERY] placeholder accepts the raw user question, while [SESSION_CONTEXT] should contain a brief summary of prior turns if available. The [CONSTRAINTS] field is where you inject latency budgets, cost limits, or access-control rules. The [EXAMPLES] placeholder should contain one or two worked examples showing correct capability-to-backend mappings for your domain. After generating the plan, validate the output against your actual backend schemas before routing queries—the model may hallucinate capabilities that don't exist. For production use, add a post-generation validation step that checks every backend_id against a real registry and rejects plans referencing unavailable backends.

This prompt is designed to sit inside a query planning service that runs before retrieval execution. It should be called when the system needs to decide which backends (dense vector, sparse keyword, graph, hybrid) are capable of answering a given user query, given the available indexes and their documented capabilities. The prompt is not a real-time retrieval step—it is a planning step that produces a capability-to-requirement mapping used by downstream orchestrators to route queries and select backends. Expect to run this prompt once per query type or per new backend configuration, not per user request, unless backend capabilities change dynamically.

Integration pattern: Wrap the prompt in a service function that accepts a [BACKEND_CAPABILITY_MANIFEST] (a structured description of each available backend, its index type, supported operations, latency profile, and schema constraints) and a [QUERY_REQUIREMENTS] object (the user's question plus any extracted constraints like time ranges, entity types, or expected answer format). The function should call the LLM, parse the JSON output against a strict schema that includes backend_assignments, capability_gaps, and confidence_scores, and validate that every assigned backend actually exists in the input manifest. Model choice: Use a model with strong JSON-following behavior and reasoning capabilities—GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro are appropriate. Avoid smaller models for this task because capability reasoning requires holding the full manifest in context and making non-obvious trade-off decisions.

Validation and retry logic: After parsing the LLM output, run a validator that checks: (1) all backend_id values match an entry in the input manifest, (2) no backend is assigned a query requirement that its manifest explicitly marks as unsupported, (3) every required query capability has at least one backend assigned, and (4) capability_gaps entries are non-empty only when genuine gaps exist. If validation fails, construct a retry prompt that includes the original input, the invalid output, and a specific error message describing which validation rule failed. Retry once. If the second attempt also fails, log the failure, fall back to a default routing rule (e.g., send all queries to the most general-purpose backend), and alert the operations team. Logging: Log the full prompt input, output, validation result, and any retry attempts to your observability platform. Tag logs with prompt_id, backend_manifest_version, and query_fingerprint for traceability.

Human review gates: For production systems where backend misrouting causes user-visible failures, add a human review step when the prompt reports confidence_score below a threshold (e.g., < 0.7) or when capability_gaps contains entries that affect critical query types. The review interface should show the user query, the capability manifest, the model's assignment, and the gap explanation side by side. What to avoid: Do not call this prompt on every user query in a high-throughput system—cache capability assignments by query pattern or intent class. Do not use this prompt to generate actual retrieval queries; it produces backend assignments, not query strings. Downstream rewriters handle query formulation per backend. Finally, keep the backend capability manifest updated whenever indexes change, or the prompt will make decisions based on stale information.