Context Completeness Scoring Prompt Template

This prompt is a pre-generation diagnostic tool for RAG pipeline architects and search engineers who need to monitor retrieval quality at scale. Its job is to produce a structured, multi-dimensional scorecard that tells you whether the retrieved context is complete enough to answer a user query. Use it when you need a numerical completeness score with dimension-level breakdowns for factual coverage, temporal relevance, authority, and specificity. The output is designed to feed directly into retrieval quality dashboards, A/B test evaluation frameworks, and continuous monitoring pipelines. It is not a replacement for answer generation or hallucination detection—it operates upstream, before the model attempts to produce an answer.

The prompt expects two inputs: a user query and a set of retrieved passages. It returns a structured scorecard with an overall completeness score, dimension-level subscores, and a binary sufficiency flag. Wire this into your retrieval evaluation harness by running it against sampled query-retrieval pairs from production logs. Store the scores alongside retrieval metadata (retriever version, index snapshot, embedding model) to enable A/B comparisons. For high-stakes applications, set a sufficiency threshold below which the system should trigger re-retrieval, clarification requests, or human review rather than proceeding to answer generation. The prompt works best with retrieval sets of 3-10 passages; larger sets may require summarization or pre-filtering to stay within context windows.

Do not use this prompt as a user-facing response or as a substitute for answer grounding verification. It evaluates retrieval completeness, not answer faithfulness. If you need to verify that a generated answer is supported by evidence, pair this with a grounding verification prompt from the Answer Grounding and Faithfulness Verification playbook. Start by running this prompt against a labeled evaluation set where you know the ground-truth completeness to calibrate thresholds before deploying it as a production gate. Avoid using it for open-ended creative queries where completeness is inherently subjective—this prompt is designed for fact-seeking queries with verifiable information needs.

This prompt template produces a structured completeness scorecard for a retrieval set against a user query. It evaluates four dimensions—factual coverage, temporal relevance, authority, and specificity—and returns a numerical score for each, plus an overall composite score. The output is designed to feed directly into retrieval quality monitoring dashboards, A/B test evaluations, and pre-generation guardrails. Replace every square-bracket placeholder with your own data before running the prompt in your evaluation harness.

textCopy
You are a retrieval quality evaluator. Your task is to assess how completely a set of retrieved passages covers the information needed to answer a user query.

## INPUT
Query: [QUERY]
Retrieved Passages:
[PASSAGE_1]
[PASSAGE_2]
[PASSAGE_3]
...

## EVALUATION DIMENSIONS
Score each dimension from 0.0 (completely insufficient) to 1.0 (fully sufficient).

1. Factual Coverage: Do the passages contain the facts, entities, and relationships needed to answer the query?
2. Temporal Relevance: Are the passages current enough for the query's time sensitivity? Flag if dates are missing or outdated.
3. Authority: Do the passages come from credible, appropriate sources for this query domain?
4. Specificity: Do the passages provide detail at the right level of granularity, or are they too vague/general?

## CONSTRAINTS
- Only use information present in the provided passages. Do not bring in external knowledge.
- If a dimension cannot be assessed due to missing metadata, score it as 0.0 and note the reason.
- If the query has no explicit time requirement, score Temporal Relevance as 0.5 and note the assumption.
- Flag any passages that contradict each other.

## OUTPUT SCHEMA
Return valid JSON matching this schema:
{
  "overall_completeness_score": <float 0.0-1.0>,
  "dimension_scores": {
    "factual_coverage": <float>,
    "temporal_relevance": <float>,
    "authority": <float>,
    "specificity": <float>
  },
  "missing_information": ["<string describing a specific gap>"],
  "contradictions": ["<string describing conflicting passages>"],
  "assessment_notes": "<string summarizing key findings>"
}

## RISK LEVEL
[RISK_LEVEL]

## EXAMPLES
[EXAMPLES]

Adaptation notes: If your retrieval pipeline already provides source metadata such as publication dates or domain authority scores, inject those into the passage blocks as structured fields rather than relying on the model to infer them. For high-stakes domains such as healthcare or finance, set [RISK_LEVEL] to high and route outputs scoring below 0.7 to human review before any downstream answer generation. The [EXAMPLES] placeholder should contain at least two scored examples showing both sufficient and insufficient retrieval sets to calibrate the model's scoring behavior. Validate the output JSON against the schema before ingestion into monitoring systems; malformed missing_information arrays or missing dimension scores are common failure modes when the model encounters ambiguous queries.

The Context Completeness Scoring prompt is designed to be a programmatic component within a RAG evaluation harness, not a one-off manual check. Its structured scorecard output—with dimension-level breakdowns for factual coverage, temporal relevance, authority, and specificity—makes it directly ingestible by monitoring dashboards and automated quality gates. To integrate it, wrap the prompt in a function that accepts a query and a list of retrieved passages, calls your LLM with a low temperature (e.g., 0.0–0.2) for deterministic scoring, and parses the JSON response into your metrics store. The primary goal is to generate a continuous signal of retrieval quality that can be tracked over time, compared across experiments, and used to trigger alerts when scores degrade below defined thresholds.

A robust implementation requires validation and retry logic. After parsing the model's JSON output, validate that all required fields (completeness_score, dimensions, missing_aspects) are present and that numerical scores fall within the expected 0.0–1.0 range. If validation fails, implement a single retry with the error message appended to the prompt context; if it fails again, log the raw output and emit a null score to avoid poisoning your metrics. For high-stakes pipelines, route low-confidence scores (e.g., completeness_score < 0.5) to a human review queue or a secondary LLM judge for confirmation. Log every scoring event with the query, passage IDs, scores, and model version to enable traceability and regression analysis when retrieval performance shifts. This structured logging is essential for correlating completeness scores with downstream task accuracy.

When deploying this prompt in a monitoring dashboard, avoid treating the numerical score as an absolute truth. Instead, use it as a relative signal: track rolling averages, compare score distributions between production and candidate retrieval pipelines during A/B tests, and set dynamic thresholds based on historical baselines rather than arbitrary cutoffs. The dimension-level breakdown is particularly valuable for diagnosing specific failure modes—a drop in temporal_relevance might indicate a stale index, while a drop in authority could signal a crawl of low-quality sources. Do not use this prompt as a real-time gate before answer generation unless you have measured and accepted the added latency; it is best suited for asynchronous evaluation, nightly regression runs, and continuous monitoring of retrieval quality in production.