Threshold-Based Evidence Filtering Prompt

This prompt is designed for RAG system operators and pipeline engineers who need to programmatically exclude low-quality evidence before it reaches the answer generation step. It applies a configurable score threshold to a set of pre-scored passages, filters out those that fall below the cutoff, and produces explicit, auditable reasoning for every exclusion decision. The primary job-to-be-done is preventing noise, irrelevant context, and weak evidence from degrading downstream answer quality, while maintaining a transparent record of what was removed and why. The ideal user is someone who already has a retrieval pipeline that assigns relevance or quality scores to passages—this prompt is a deterministic filtering gate, not a scoring engine.

Use this prompt when your retrieval pipeline produces numerical scores for each passage, you have a defined quality threshold, and you need an auditable filtering step before answer synthesis. It is particularly valuable in high-stakes domains like legal research, clinical decision support, or financial analysis where excluding weak evidence is as important as including strong evidence. The prompt expects a structured input containing scored passages with their associated scores and a threshold value. It returns a filtered set of passages that meet or exceed the threshold, along with a justification for each excluded passage that explains why the score was insufficient. This creates an audit trail that downstream systems and human reviewers can inspect.

Do not use this prompt when you need to generate scores for un-scored passages—it assumes scores already exist and focuses solely on thresholding logic. It is also not suitable for cases where you need nuanced re-ranking of passages that all exceed the threshold, or where the threshold itself needs to be dynamically calculated from the score distribution. For those scenarios, consider pairing this prompt with an evidence scoring prompt from the Evidence Weighting and Scoring Models group, or a re-ranking prompt that handles relative ordering. Additionally, avoid this prompt when the filtering decision requires subjective judgment about content quality beyond what a numerical score captures—the prompt's reasoning is constrained to explaining threshold failures, not performing deep content analysis.

Before deploying this prompt in production, establish clear evaluation criteria for your threshold setting. Over-filtering removes useful context and can cause downstream answer generation to miss critical evidence, while under-filtering admits noise that degrades answer quality. Run this prompt against a golden dataset of scored passages with known good and bad examples, and measure both precision (are excluded passages truly low-quality?) and recall (are all low-quality passages being caught?). In regulated or high-risk domains, always route exclusion decisions above a certain impact threshold to human review, and log all filtering decisions for audit purposes.

This prompt template implements threshold-based evidence filtering for RAG pipelines. It accepts a set of scored passages, applies minimum score thresholds per factor (relevance, recency, authority, specificity), and returns only the passages that meet all criteria. Each excluded passage receives an explicit reason, and the prompt flags borderline cases where threshold adjustments might be warranted. Use this when downstream answer generation depends on clean, high-quality evidence and when exclusion decisions must be auditable.

codeCopy
You are an evidence filtering system for a RAG pipeline. Your job is to apply configurable score thresholds to a set of retrieved passages and determine which passages qualify for downstream answer generation.

## INPUT
- Passages with pre-computed scores: [SCORED_PASSAGES]
- Threshold configuration: [THRESHOLD_CONFIG]
- Query context: [USER_QUERY]

## THRESHOLD CONFIGURATION
Apply these minimum score thresholds to each passage:
- relevance_min: [RELEVANCE_THRESHOLD]
- recency_min: [RECENCY_THRESHOLD]
- authority_min: [AUTHORITY_THRESHOLD]
- specificity_min: [SPECIFICITY_THRESHOLD]

## INSTRUCTIONS
1. For each passage, check whether ALL factor scores meet or exceed their respective minimum thresholds.
2. If a passage meets ALL thresholds, mark it as INCLUDED.
3. If a passage fails ANY threshold, mark it as EXCLUDED and provide a specific reason naming the failed factor(s) and the actual score versus the threshold.
4. For passages within [MARGIN] points of any threshold, add a BORDERLINE flag with a note explaining the risk of over-filtering or under-filtering.
5. Do not modify, summarize, or re-score the passages. Only filter and annotate.

## OUTPUT FORMAT
Return a JSON object with this exact structure:
{
  "filtered_passages": [
    {
      "passage_id": "string",
      "status": "INCLUDED" | "EXCLUDED",
      "scores": {
        "relevance": number,
        "recency": number,
        "authority": number,
        "specificity": number
      },
      "exclusion_reason": "string | null",
      "borderline_flag": {
        "is_borderline": boolean,
        "near_threshold_factors": ["string"],
        "risk_note": "string | null"
      }
    }
  ],
  "summary": {
    "total_passages": number,
    "included_count": number,
    "excluded_count": number,
    "borderline_count": number,
    "filtering_concern": "string | null"
  }
}

## CONSTRAINTS
- Do not invent scores. Use only the scores provided in [SCORED_PASSAGES].
- If a passage is missing a required score factor, treat that factor as 0 and exclude the passage with reason "missing_score".
- If fewer than [MIN_PASSAGES_TO_RETAIN] passages qualify, include a filtering_concern in the summary noting "insufficient_qualifying_passages" and list the highest-scoring excluded passages as candidates for threshold relaxation.
- If more than [MAX_PASSAGES_TO_RETAIN] passages qualify, keep only the top [MAX_PASSAGES_TO_RETAIN] by composite score and note the truncation in the summary.

Adaptation guidance: Replace [SCORED_PASSAGES] with your retrieval output including pre-computed factor scores. Set [THRESHOLD_CONFIG] thresholds based on your domain calibration—start conservatively and tighten after reviewing borderline flags in production. The [MARGIN] parameter controls how many near-threshold passages get flagged for review; 0.05–0.10 is a reasonable starting range on a 0–1 scale. Wire [MIN_PASSAGES_TO_RETAIN] and [MAX_PASSAGES_TO_RETAIN] to your downstream answer generator's context window and quality requirements. If your scoring system uses different factor names, replace the factor keys throughout the template and output schema. Always validate the JSON output against the schema before passing filtered passages to answer generation—malformed exclusion reasons or missing borderline flags indicate the model misunderstood the task and the filtering results should not be trusted.

The threshold-based evidence filtering prompt is designed to sit between retrieval and answer generation in a RAG pipeline. It accepts a ranked list of evidence passages with scores and a configurable threshold, then returns only the passages that meet or exceed that threshold along with explicit reasoning for each exclusion. In production, this prompt should be called as a synchronous step after retrieval re-ranking and before the final answer synthesis prompt. The primary integration points are: (1) a retrieval service that provides scored passages, (2) a threshold configuration store that can be adjusted per query type or risk level, (3) the filtering prompt itself, and (4) downstream answer generation that receives only filtered evidence. Avoid calling this prompt when the retrieval set is already small — filtering three passages down to one may discard useful context that a capable model could have navigated with appropriate uncertainty expression.

Validation is the first line of defense after the model returns. Parse the output against the expected schema: a list of kept passages, a list of excluded passages, and a reason string for each exclusion. Validate that every passage in the input set appears exactly once across the kept and excluded lists — missing passages indicate a model output error. Check that excluded passages have scores strictly below the threshold and kept passages have scores at or above it. If the model excluded a passage above threshold or kept one below, flag it for retry. For high-stakes domains such as healthcare or legal, add a human review step when the filtering decision is borderline — specifically when a passage score falls within 0.05 of the threshold on a normalized 0–1 scale. Log the full input set, threshold value, model output, and validation result for every call. This trace data is essential for debugging threshold calibration issues later.

Retry logic should be bounded and diagnostic. If validation fails, retry once with the same prompt but append the validation errors as additional context in a [VALIDATION_ERRORS] placeholder, instructing the model to correct the specific discrepancies. If the second attempt also fails validation, fall back to a simple score-threshold cutoff without model reasoning — this ensures the pipeline degrades gracefully rather than blocking answer generation entirely. For model choice, prefer a fast, instruction-following model such as GPT-4o-mini, Claude Haiku, or Gemini Flash for this filtering step. The task is classification and justification, not complex reasoning, and latency matters because this step adds overhead between retrieval and answer generation. Avoid using the same model instance for filtering and answer generation if throughput is a concern — parallelize where possible.

Observability is critical because threshold tuning is an empirical process. Track three metrics in production: the filtering rate (percentage of passages excluded per query), the threshold boundary density (how many passages fall within 0.1 of the threshold), and downstream answer grounding scores. If the filtering rate is consistently above 80%, your retrieval set may be too noisy or your threshold is too aggressive — investigate retrieval quality before adjusting the threshold. If answer grounding scores degrade after introducing filtering, you may be over-filtering and removing context the answer model needs. Set up a dashboard that plots these metrics per query category so you can tune thresholds independently for different use cases. Finally, run periodic eval sweeps using a golden dataset of query–passage–threshold triples with known good filtering decisions to detect drift in model behavior across model version updates.