Context Adequacy Evaluation Prompt Template

This prompt is a production-ready adequacy gate for RAG pipelines. Its job is to evaluate whether retrieved context is sufficient for a specific use case or decision type before answer generation begins. Pipeline operators and AI engineers use it to prevent low-quality answers by stopping the pipeline when evidence is inadequate. The prompt requires a domain-specific adequacy rubric configured in the harness layer. Use this when you need a structured pass/fail decision with detailed adequacy dimensions rather than a simple confidence score.

The ideal user is an AI engineer or pipeline operator who owns a RAG system and needs a deterministic guard between retrieval and generation. This prompt is not a post-generation factuality check, nor is it designed for ranking individual passages. It belongs in the critical path where a decision must be made: proceed to answer generation, trigger additional retrieval, or escalate to a human. The harness must supply a rubric that defines what 'adequate' means for your domain—for example, a legal use case might require specific clause coverage, while a customer support use case might require entity presence and temporal relevance.

Do not use this prompt when you need a lightweight confidence score, when you are evaluating an already-generated answer, or when your retrieval pipeline cannot support a re-retrieval or escalation path. If your system has no fallback behavior for an 'inadequate' decision, this gate adds latency without value. Start by defining your adequacy rubric, then integrate this prompt between retrieval and answer generation with clear routing logic for each possible output.

This template is designed to act as an adequacy gate in your RAG pipeline. It forces the model to assess retrieved context against a domain-specific rubric before answer generation proceeds. The prompt expects you to define what 'adequate' means for your use case through the [ADEQUACY_RUBRIC] placeholder—this is where you encode your domain's evidence standards, required entity coverage, temporal constraints, and source authority requirements. Without a well-defined rubric, the gate becomes arbitrary and will produce inconsistent pass/fail decisions across similar inputs.

textCopy
You are a context adequacy evaluator for a retrieval-augmented generation pipeline. Your job is to determine whether the retrieved context is sufficient to answer the user's question for the specified use case, following a strict adequacy rubric.

## INPUTS

**User Question:**
[USER_QUESTION]

**Retrieved Context:**
[RETRIEVED_CONTEXT]

**Use Case:**
[USE_CASE]

**Adequacy Rubric:**
[ADEQUACY_RUBRIC]

## OUTPUT SCHEMA

Return a JSON object with exactly these fields:

{
  "adequacy_decision": "PASS" | "FAIL",
  "adequacy_score": <float between 0.0 and 1.0>,
  "dimensions": {
    "factual_coverage": {
      "score": <float 0.0-1.0>,
      "assessment": "<explanation of what facts are covered and what is missing>"
    },
    "temporal_relevance": {
      "score": <float 0.0-1.0>,
      "assessment": "<explanation of whether context timestamps match question timeframe>"
    },
    "source_authority": {
      "score": <float 0.0-1.0>,
      "assessment": "<explanation of source credibility and relevance to question domain>"
    },
    "entity_completeness": {
      "score": <float 0.0-1.0>,
      "assessment": "<explanation of whether all key entities from question appear in context>"
    },
    "rubric_alignment": {
      "score": <float 0.0-1.0>,
      "assessment": "<explanation of how context satisfies or violates the adequacy rubric>"
    }
  },
  "missing_information": [
    "<specific fact, entity, or evidence type that is absent from context>"
  ],
  "recommendation": "<PROCEED_WITH_ANSWER | RETRIEVE_MORE | ESCALATE_TO_HUMAN | FLAG_AS_UNANSWERABLE>",
  "rationale": "<concise explanation of the adequacy decision, referencing rubric criteria>"
}

## CONSTRAINTS

- Base your assessment ONLY on the retrieved context provided. Do not use external knowledge.
- If the context contains contradictory information, note this in the rationale and factor it into the adequacy score.
- A PASS decision requires all rubric criteria to be met at an acceptable threshold. Partial coverage that fails any critical criterion must result in FAIL.
- The adequacy_score must reflect the weighted importance of rubric criteria, not a simple average of dimension scores.
- If the question requires information from multiple documents and only one is present, flag this in missing_information.
- For [RISK_LEVEL] use cases, err toward FAIL when evidence is ambiguous or incomplete.

Adaptation guidance: Replace [ADEQUACY_RUBRIC] with concrete, testable criteria specific to your domain. For example, a clinical rubric might require 'evidence from guidelines published within the last 2 years' and 'explicit mention of contraindications.' A legal rubric might require 'citation to primary sources, not summaries' and 'jurisdiction match.' The rubric is the engine of this prompt—invest time defining it with domain experts and validate it against a golden dataset of known-adequate and known-inadequate context pairs. The [RISK_LEVEL] placeholder should be set to HIGH, MEDIUM, or LOW to adjust the evaluator's conservatism. Wire the output recommendation field into your pipeline's routing logic: PROCEED_WITH_ANSWER triggers generation, RETRIEVE_MORE triggers an additional retrieval round, and ESCALATE_TO_HUMAN or FLAG_AS_UNANSWERABLE routes to a review queue or user-facing abstention message.

This prompt operates as a pre-generation gate in a RAG pipeline. It should be called after retrieval but before answer synthesis. The harness receives retrieved chunks and the user query, injects them into the [RETRIEVED_CONTEXT] and [USER_QUERY] placeholders, and expects a structured JSON response containing a pass/fail decision and adequacy dimensions. The output determines whether the pipeline proceeds to answer generation, triggers re-retrieval, or escalates to a human queue.

Validation is mandatory. Parse the model's JSON output and validate that the pass field is a boolean, adequacy_dimensions contains all required keys from your rubric, and each dimension score is within the expected numeric range. If validation fails, retry once with the same context and an explicit error message appended to the prompt: Your previous output failed schema validation. Ensure 'pass' is boolean and all adequacy dimensions are present with numeric scores. After two failures, log the raw output and route to a human review queue. Model choice matters: use a model with strong instruction-following and JSON mode support (e.g., GPT-4o, Claude 3.5 Sonnet) for this gate. Smaller models often produce malformed dimension scores or hallucinate adequacy claims.

Domain-specific rubric injection is the most critical integration point. The [ADEQUACY_RUBRIC] placeholder must be populated from a configuration store, not hardcoded. For a legal document review use case, dimensions might include authority_of_source, jurisdictional_relevance, and temporal_validity. For customer support, dimensions might include symptom_coverage, resolution_steps_present, and product_version_match. Store these rubrics as versioned JSON objects in your application config. Log every evaluation with the query hash, context chunk IDs, rubric version, model response, and routing decision. This audit trail is essential for debugging false-positive adequacy claims and tuning rubric thresholds over time.

Threshold tuning requires offline evaluation. Before deploying, run a golden dataset of 100+ query-context pairs with known adequacy labels through the harness. Plot the distribution of composite scores for adequate vs. inadequate pairs. Set your pass threshold at the score that maximizes F1 for your use case's tolerance for false positives (generating from inadequate context) vs. false negatives (unnecessary re-retrieval). For high-stakes domains like clinical or legal, bias toward false negatives—re-retrieval is cheaper than an incorrect answer. Do not rely on the model's internal pass boolean alone; compute the composite score from dimension values in your application code and apply the threshold there for consistent, auditable decisions.

Integration with retrieval loops is the next step. When the gate returns pass: false, the harness should trigger a retrieval expansion strategy: rewrite the query using a separate query-rewriting prompt, broaden search parameters, or query alternative indexes. After re-retrieval, run this adequacy evaluation again. Set a maximum of 2-3 retrieval rounds before escalating. Avoid infinite loops by tracking retrieval attempt count per query and forcing escalation when exceeded. For systems with human-in-the-loop fallback, route pass: false after max retries to a review interface that displays the original query, retrieved context, and adequacy dimension scores so a human can decide whether to answer, refine retrieval, or mark the question as unanswerable.