Context Pruning Prompt for Irrelevant Evidence

This prompt is designed for RAG system operators and AI engineers who need to reduce context window bloat by removing irrelevant retrieved passages before they reach the model. The core job is classification and pruning: given a user query and a set of retrieved passages, the prompt classifies each passage as essential, supportive, or irrelevant to the query, drops the irrelevant tier, and returns a pruned context set with brief removal justifications. This is not a summarization prompt—it preserves the original text of retained passages verbatim. Use it when your retrieval pipeline returns loosely related content alongside critical evidence, and you need to control token costs and reduce distraction without losing fidelity on what matters.

The ideal user is an engineer or operator who already has a working retrieval step and wants to insert a pruning stage before the final generation call. You need three inputs: a clear user query or task description, a set of retrieved passages with source identifiers, and a defined output schema for the pruned result. The prompt works best when passages are self-contained enough to be classified independently. It is not suitable when passages are deeply interdependent, when the task requires comparing passages against each other, or when the definition of relevance depends on subtle domain knowledge the model cannot infer from the query alone. In those cases, use an evidence ranking prompt or a contradiction resolution prompt instead.

Before deploying this prompt in production, define what 'irrelevant' means for your use case and test against a golden set of passages with known relevance labels. The prompt includes a structured output schema and removal justifications, which makes it straightforward to evaluate with LLM judges or human reviewers. For high-stakes domains—legal, clinical, financial—require human approval on the pruned set before it feeds downstream generation. The biggest failure mode is over-pruning: the model drops a passage that contains critical context not obvious from the query alone. Mitigate this by running a factuality preservation check on the pruned context against the original, and by setting a minimum retention threshold when the task involves compliance or audit requirements.

This template is the core instruction set for the context pruning workflow. It takes a user query, a set of retrieved passages, and a task description, then classifies each passage as essential, supportive, or irrelevant to the current task. The prompt outputs a pruned context set containing only essential and supportive passages, along with removal justifications for every dropped passage. Use this template when retrieval returns loosely related or noisy results that would waste token budget, dilute model attention, or introduce contradictory signals into downstream reasoning.

textCopy
You are a context pruning specialist. Your job is to evaluate each retrieved passage against the user's query and the stated task, then remove passages that do not contribute useful information.

## INPUT

**User Query:**
[USER_QUERY]

**Task Description:**
[TASK_DESCRIPTION]

**Retrieved Passages:**
[RETRIEVED_PASSAGES]

## CLASSIFICATION RULES

For each passage, assign exactly one classification:

- **essential**: The passage contains information that is directly required to answer the query or complete the task. Removing it would make the task impossible or significantly degrade correctness.
- **supportive**: The passage provides useful context, background, or corroborating evidence that improves answer quality but is not strictly required. The task can be completed without it, though quality may suffer.
- **irrelevant**: The passage does not help answer the query or complete the task. It may be about a different topic, too vague, redundant with a better passage, or contain only noise.

## CONSTRAINTS

[CONSTRAINTS]

## OUTPUT FORMAT

Return a JSON object with this exact structure:

{
  "pruned_passages": [
    {
      "passage_id": "string",
      "classification": "essential | supportive",
      "passage_text": "string"
    }
  ],
  "removed_passages": [
    {
      "passage_id": "string",
      "classification": "irrelevant",
      "removal_reason": "string explaining why this passage was dropped"
    }
  ],
  "pruning_summary": {
    "total_input_passages": number,
    "essential_count": number,
    "supportive_count": number,
    "irrelevant_count": number,
    "token_savings_estimate": "string describing approximate token reduction"
  }
}

## EXAMPLES

[EXAMPLES]

## IMPORTANT

- Do not modify passage text. Return passages exactly as provided.
- If all passages are irrelevant, return an empty pruned_passages array and explain why in the pruning_summary.
- If you are uncertain about a passage's relevance, classify it as supportive rather than dropping it.
- Preserve passage_id values exactly as provided in the input.

To adapt this template, populate the square-bracket placeholders with your runtime data. [USER_QUERY] receives the original user question or instruction. [TASK_DESCRIPTION] describes what the downstream model will do with the pruned context—be specific about the output type, audience, and quality bar so the pruning classifier can make accurate relevance judgments. [RETRIEVED_PASSAGES] should be a JSON array of objects with passage_id and passage_text fields. Use [CONSTRAINTS] to inject domain-specific rules such as 'retain all passages mentioning regulatory requirements' or 'drop passages older than 2023 unless explicitly cited.' Provide 2-3 [EXAMPLES] showing correct classification decisions for your domain, including edge cases where a passage appears relevant but is actually redundant or outdated. Before deploying, validate that the output JSON parses correctly and that every input passage_id appears exactly once across the pruned and removed arrays.

The context pruning prompt is not a standalone utility; it is a pipeline stage that sits between retrieval and answer generation. The typical call pattern is: retrieve N passages, assemble them into the [CONTEXT] placeholder, invoke the pruning prompt, validate the output, and pass only the essential and supportive passages to the downstream generation prompt. This stage should be treated as a deterministic filter with a structured output contract—not an open-ended chat completion. Use a low-temperature setting (0.0–0.2) and a model that reliably produces valid JSON, such as gpt-4o or claude-3-5-sonnet, to minimize variance in classification decisions.

Validation and retry logic must be built around the expected output schema. Before the pruned context reaches the answer generator, validate that: (1) the response is parseable JSON matching the pruned_context schema; (2) every passage ID in the output exists in the input set; (3) no passage appears in more than one tier; and (4) the removal_justifications array has an entry for every irrelevant passage. If validation fails, retry once with the same prompt and a slightly higher temperature (0.3) to jitter the output. If the second attempt fails, log the failure, fall back to passing all passages to the answer generator, and flag the request for review. For high-stakes domains, add a human review gate on any batch where more than 30% of retrieved passages are classified as irrelevant, as this may indicate a retrieval quality problem rather than a pruning problem.

Observability is critical because pruning decisions directly affect downstream answer quality. Log the full input passage set, the pruning output, the token count before and after pruning, and the latency of the pruning call. Attach a trace ID that links the retrieval, pruning, and generation stages so operators can diagnose whether a bad answer originated from poor retrieval, overly aggressive pruning, or a generation failure. If your pipeline uses a feature flag or A/B testing framework, wrap the pruning stage so you can compare answer quality with and without pruning in production. Start with a conservative pruning threshold—only drop passages classified as irrelevant with high confidence—and tighten only after reviewing pruning decisions against human judgments on a labeled eval set.