Top-K Evidence Selection Prompt Template

This prompt is for RAG system builders and search engineers who must fit the most valuable evidence into a constrained context window. When your retrieval pipeline returns 20, 50, or 100 passages but your answer-generation model can only accept 4K tokens of context, you need a selection step that maximizes coverage while eliminating redundancy. This prompt operates after retrieval and before answer generation, acting as a budget-aware gate that picks the top-K passages most likely to produce a grounded, complete answer. Use this when your context budget is fixed, your retrieval set is larger than your budget allows, and you need explicit coverage verification before downstream generation.

The ideal deployment scenario has three characteristics: a known token or passage-count budget, a retrieval set that exceeds that budget by at least 2x, and a downstream generator that will faithfully use only the selected passages. The prompt template accepts a ranked or unranked list of candidate passages, a query or claim to ground against, a selection budget (K), and optional constraints like diversity requirements or source authority preferences. It returns exactly K passages with selection rationale and a coverage assessment that flags any query aspects not addressed by the selected set. This coverage check is the key differentiator from simpler relevance scoring—it prevents the common failure mode where all top-K passages address the same sub-topic while leaving other parts of the question unanswered.

Do not use this prompt when your retrieval set already fits within the context budget, when you need real-time sub-50ms selection latency, or when your downstream generator has its own built-in attention mechanisms that handle long contexts natively. For retrieval sets under 10 passages, direct inclusion is simpler and avoids an extra model call. For latency-critical paths, consider embedding-based similarity ranking or a lightweight cross-encoder instead. This prompt is also inappropriate when the evidence requires complex multi-hop reasoning to assess—the selection step should evaluate individual passage value, not solve the full reasoning chain. If your use case involves regulated domains like healthcare or legal, add a human review step between selection and answer generation, and log the full selection rationale for audit trails.

This template is designed to be pasted directly into your system prompt or as a user message in a RAG pipeline. It instructs the model to act as a strict evidence curator, selecting a fixed number of passages that maximize informational value while respecting a hard context budget. The prompt forces explicit trade-offs between relevance, redundancy, and coverage, making the selection logic auditable rather than a black box. Before using it, ensure you have already retrieved a candidate set of passages and have a clear understanding of your downstream task's token limit.

textCopy
You are an evidence curator for a retrieval-augmented generation system. Your task is to select the top [K] most valuable evidence passages from the provided candidate set. You must operate within a strict total token budget of [MAX_TOKENS] tokens for the selected passages combined.

## INPUT
**User Query:** [QUERY]
**Candidate Passages:** A list of [N] passages, each with a unique ID, source metadata, and text content.
[CANDIDATE_PASSAGES]

## SELECTION CRITERIA
Rank and select passages based on the following weighted priorities:
1. **Direct Relevance (Highest Priority):** The passage must directly address a core aspect of the user query. Penalize tangential or merely topically related content.
2. **Specificity and Factual Density:** Prefer passages with concrete facts, numbers, or specific claims over vague or general statements.
3. **Non-Redundancy:** If two passages convey the same information, select only the one with the highest authority or specificity. Do not waste the budget on duplicates.
4. **Coverage:** The final set of [K] passages should collectively cover as many distinct aspects of the query as possible. Identify and flag any critical query sub-topics that have zero supporting passages.
5. **Source Authority:** When choosing between equally relevant passages, prefer those from more authoritative or recent sources based on the provided metadata.

## CONSTRAINTS
- You MUST select exactly [K] passages. No more, no less.
- The combined token count of the selected passages MUST NOT exceed [MAX_TOKENS].
- You MUST NOT modify, summarize, or paraphrase the passage text. Output the full original text of each selected passage.
- If fewer than [K] passages meet a minimum relevance threshold, select only those that do and explain the shortfall.

## OUTPUT FORMAT
Return a valid JSON object with the following schema:
{
  "selected_passages": [
    {
      "passage_id": "string",
      "rank": integer,
      "relevance_score": float (0.0 to 1.0),
      "selection_rationale": "One-sentence explanation of why this passage was chosen.",
      "text": "Full original passage text."
    }
  ],
  "coverage_assessment": {
    "covered_aspects": ["list of query sub-topics covered"],
    "missed_aspects": ["list of query sub-topics with no supporting evidence"],
    "redundancy_removed": ["list of passage IDs excluded due to redundancy"]
  },
  "budget_utilization": {
    "total_tokens_used": integer,
    "budget_remaining": integer
  }
}

To adapt this template for your application, replace the square-bracket placeholders with your runtime data. The [CANDIDATE_PASSAGES] placeholder should be populated with a pre-serialized list of your retrieval results, each containing an ID, text, and any available metadata like date or source domain. The [K] and [MAX_TOKENS] values should be derived from your downstream model's context window and the space reserved for instructions and the final answer. If your use case is high-stakes—such as medical or legal reasoning—add a [RISK_LEVEL] field and a constraint requiring the model to flag any selected passage that contains speculative or unverified claims. Always validate the output JSON against the schema before passing the selected passages to your answer generation step. A common failure mode is the model returning K-1 or K+1 passages; implement a retry loop that detects count mismatches and re-prompts with a stricter instruction.

The Top-K Evidence Selection prompt is designed to sit between retrieval and answer generation in a RAG pipeline. It consumes a larger retrieval set and produces a compact, ranked subset that fits within a strict context budget. The implementation harness must handle three core responsibilities: enforcing the token budget, validating the output structure, and providing a fallback path when the model fails to produce a usable ranking. This prompt is not a standalone component—it is a gating step that determines which evidence reaches the downstream answer generator, so harness reliability directly impacts answer quality.

Wire the prompt into your application as a pre-generation filter. After retrieval returns N passages, call the Top-K selection prompt with [RETRIEVED_PASSAGES], [QUERY], [TOP_K], and [MAX_TOKENS]. Parse the output against a strict JSON schema that requires selected_passages (array of passage IDs), selection_rationale (string), and coverage_gaps (array of strings). Implement a validation layer that checks: (1) the number of selected passages does not exceed [TOP_K], (2) all selected passage IDs exist in the input set, (3) the total token count of selected passages is under [MAX_TOKENS], and (4) the coverage_gaps field is populated when the model identifies missing sub-topics. If validation fails, retry once with the validation errors injected into the prompt as [PREVIOUS_ERRORS]. After two failures, fall back to a deterministic baseline: select the top [TOP_K] passages by retrieval score and truncate each to fit the budget.

For model choice, use a model with strong instruction-following and structured output capabilities. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are all suitable. Avoid smaller or older models that struggle with multi-constraint ranking tasks—they tend to ignore the token budget or produce malformed passage IDs. Set temperature=0 or very low (0.1) to maximize ranking stability. For logging, capture the full input retrieval set, the selected subset, the rationale, any coverage gaps flagged, validation results, and retry counts. This trace data is essential for debugging when downstream answer quality degrades—you need to know whether the problem originated in retrieval, evidence selection, or answer generation.

When integrating with a RAG pipeline, connect the coverage gap output to a re-retrieval trigger. If the model flags that no passage addresses a required sub-topic, your harness should issue a targeted retrieval query for that gap and re-run the selection prompt with the expanded set. Cap re-retrieval at one or two rounds to avoid infinite loops. For high-stakes domains (legal, medical, financial), add a human review step when coverage gaps are detected or when the model's confidence language in the rationale indicates uncertainty. The harness should also track the token utilization ratio (selected tokens / budget) to identify cases where the model is consistently under- or over-packing the context window—both patterns indicate ranking behavior that needs tuning.

Avoid wiring this prompt directly into a synchronous user-facing response path without the validation and fallback layers described above. A malformed ranking that passes garbage passage IDs to the answer generator will produce hallucinated citations that are difficult to detect downstream. Instead, treat the Top-K selection step as an internal pipeline gate with its own monitoring: track selection latency, validation failure rate, coverage gap frequency, and fallback activation rate. These metrics will tell you when your retrieval set quality, prompt design, or model choice needs adjustment before users ever see degraded answers.