Cost-Aware Passage Selection Prompt for Token Budgets

This prompt is for RAG operators and pipeline engineers who need to select the most relevant passages from a retrieval set while respecting a hard token budget. The job-to-be-done is not just ranking passages by relevance—it's making explicit trade-offs between information coverage and per-query cost. You have a fixed context window or a cost ceiling, and you need the model to justify which passages earn their tokens and which don't. The ideal user is someone running a production RAG system where token costs directly affect margins, latency, or throughput limits, and where blindly packing the top-k passages is no longer acceptable.

Use this prompt when you have already retrieved a candidate passage set and need a budget-constrained selection before answer generation. It works best when you can supply a clear token budget, a ranked or scored retrieval list, and a specific query that defines what 'relevant' means. The prompt produces a selected subset with per-passage cost-benefit justifications, making the selection logic auditable. Do not use this prompt when your retrieval set is already small enough to fit within budget, when you need binary relevance filtering without budget constraints, or when you're optimizing for recall over precision. For those cases, use the Binary Passage Relevance Filter or Passage Sufficiency Assessment prompts instead.

This prompt is not a replacement for retrieval quality. If your retriever consistently returns irrelevant passages, cost-aware selection won't fix the underlying problem—it will just produce well-justified selections of bad candidates. Pair this prompt with retrieval evaluation and relevance scoring before introducing budget constraints. The output should feed into a validation step that checks whether the selected passages still cover the query's information needs. If the budget is too tight to retain essential evidence, the prompt should surface that gap rather than silently dropping critical context. Start with a generous budget, measure relevance retention, then tighten constraints while monitoring answer quality degradation.

This prompt template is designed for production RAG operators who need to balance passage relevance against per-query token costs. It forces the model to make explicit trade-offs: selecting passages that maximize information density within a hard budget while justifying why lower-priority passages were excluded. The template expects a pre-retrieved set of passages, a token budget, and optional domain-specific constraints. Use it when your retrieval pipeline returns more context than you can afford to send to the generation model, and you need auditable selection decisions rather than opaque truncation.

textCopy
You are a cost-aware passage selector for a retrieval-augmented generation system. Your task is to select the most relevant passages from a candidate set while respecting a strict token budget. You must maximize information density and relevance, and you must explain your trade-offs.

## INPUT

**Query:** [USER_QUERY]

**Candidate Passages:**
[PASSAGE_LIST_WITH_IDS_AND_TOKEN_COUNTS]

**Token Budget:** [MAX_TOTAL_TOKENS]

**Domain Constraints (if any):** [DOMAIN_CONSTRAINTS]

## INSTRUCTIONS

1. Review each candidate passage for relevance to the query, information density, and uniqueness relative to other passages.
2. Select passages that fit within the token budget while maximizing coverage of the query's information needs.
3. Prefer passages that provide direct evidence, specific facts, or authoritative answers over vague or tangential content.
4. If two passages contain overlapping information, select the more concise or more authoritative one.
5. If the budget forces exclusion of relevant passages, prioritize those that address the core question over those that provide supplementary context.
6. Do not exceed the token budget. Count tokens conservatively.

## OUTPUT SCHEMA

Return a JSON object with the following structure:

{
  "selected_passages": [
    {
      "passage_id": "string",
      "token_count": number,
      "relevance_score": number (0.0 to 1.0),
      "selection_rationale": "string explaining why this passage was selected"
    }
  ],
  "excluded_passages": [
    {
      "passage_id": "string",
      "token_count": number,
      "relevance_score": number (0.0 to 1.0),
      "exclusion_rationale": "string explaining why this passage was excluded despite its relevance score"
    }
  ],
  "total_tokens_used": number,
  "budget_utilization_percent": number,
  "coverage_assessment": "string describing what the selected passages cover and what information gaps remain",
  "selection_summary": "string briefly explaining the overall selection strategy"
}

## CONSTRAINTS

- Total tokens of selected passages must not exceed [MAX_TOTAL_TOKENS].
- Every passage in the candidate list must appear in either selected_passages or excluded_passages.
- Relevance scores must be justified by the passage content, not the passage position in the list.
- If no passages are sufficiently relevant, return an empty selected_passages array and explain why in coverage_assessment.
- Do not fabricate passage content. Only reference information present in the provided passages.

To adapt this template for your pipeline, replace the square-bracket placeholders with your actual data. The [PASSAGE_LIST_WITH_IDS_AND_TOKEN_COUNTS] placeholder should receive a structured list where each passage includes a unique ID, the full passage text, and a pre-computed token count. Pre-compute token counts in your application layer using your target model's tokenizer rather than relying on the model to estimate them. The [DOMAIN_CONSTRAINTS] field is optional but valuable for specialized deployments—for example, a medical RAG system might add "Prefer passages from peer-reviewed journals published within the last 5 years" or a legal system might add "Exclude passages from dissenting opinions unless the query specifically asks about dissents." If your pipeline uses a multi-stage filtering approach, you can chain this prompt after a cheaper binary relevance filter to reduce the candidate set before budget-constrained selection. Always validate the output JSON against the schema before passing selections to your generation step, and log exclusion rationales for pipeline debugging.

This prompt operates as a pre-generation filter in your RAG pipeline. After retrieval returns a candidate passage set, but before those passages are packed into the final context window for answer generation, this prompt selects the optimal subset that maximizes relevance while respecting a hard token budget. The implementation harness must enforce the budget constraint programmatically, validate the model's output against that constraint, and provide fallback behavior when the model fails to produce a valid selection. Do not rely on the model alone to count tokens accurately—always verify the selected passages' token count in your application code using the same tokenizer your target model uses.

Wiring the prompt into your application requires three layers: a pre-call layer, a post-call validation layer, and a fallback layer. In the pre-call layer, compute the token count for each candidate passage using tiktoken (for OpenAI models) or the equivalent tokenizer for your model family. Pass these counts alongside the passages in the [PASSAGES] placeholder as structured objects: {"id": "p1", "text": "...", "token_count": 145}. The [BUDGET] placeholder should receive an integer representing the maximum tokens allowed for the selected set. In the post-call layer, parse the model's JSON output, extract the selected passage IDs, sum their pre-computed token counts, and reject any selection that exceeds the budget. If validation fails, retry once with an explicit error message injected into the prompt: "Your previous selection used {actual_tokens} tokens, exceeding the budget of {budget} tokens. Select fewer or shorter passages." After two failures, fall back to a deterministic greedy selection algorithm that picks the highest-scored passages from the initial retrieval ranking until the budget is exhausted.

Logging and observability are critical because this prompt directly trades off answer quality against cost. Log the following per request: the initial retrieval set size, the budget constraint, the selected passage IDs and their individual token counts, the total tokens used, the number of retries, and whether the fallback algorithm was invoked. Over time, these logs reveal whether your budget is consistently too tight (frequent fallback invocation, degraded answer quality) or too loose (unused budget that could have included more evidence). Wire these metrics into your existing monitoring stack and set alerts for fallback rates exceeding 5% of requests. For high-stakes applications, consider an A/B evaluation harness that runs the same query through the cost-aware selection and an unbounded selection, then compares answer quality using an LLM judge to quantify the quality-cost trade-off at different budget levels.