Retrieval-Augmented Classification Prompt with Evidence Support

This prompt is designed for classification tasks where every label assignment must be justified by retrieved evidence. Use it when you need a model to classify an input, cite the specific passages that support the classification, and provide a confidence score. This pattern is essential for audit-heavy domains like compliance, legal intake, medical coding, and financial document review, where a label without a source is a liability. The prompt assumes you have already retrieved a set of candidate passages from a vector store, search index, or knowledge base. It does not perform retrieval itself; it consumes retrieval results and produces a structured, evidence-backed classification.

The ideal user is a product engineer or AI builder integrating classification into a system of record. You should use this prompt when downstream actions depend on the label, when a reviewer needs to verify the decision later, or when the cost of a wrong label is high. The required inputs are a clear classification schema with label definitions, the raw text to classify, and a set of retrieved evidence passages. Without well-defined labels and relevant evidence in the context window, the prompt will produce unreliable output. Do not use this pattern for real-time, low-latency user-facing features where retrieval adds unacceptable delay, or for subjective sentiment tasks where evidence is inherently ambiguous.

Before adopting this prompt, verify that your retrieval step consistently returns relevant passages. If your vector search or keyword index is immature, the model will be forced to classify with poor evidence, leading to low confidence scores or incorrect citations. Start by running this prompt against a golden dataset of 50-100 labeled examples and measure both classification accuracy and citation correctness. If citation accuracy is below your threshold, invest in retrieval quality before tuning the prompt. For high-risk domains, always route low-confidence outputs to a human review queue rather than accepting the model's best guess.

This template is the core instruction block for a retrieval-augmented classification system. It forces the model to pair each predicted label with supporting evidence extracted directly from the provided context, rather than relying on parametric knowledge. The structure separates the classification task from the evidence-justification task, making it easier to validate outputs programmatically. Use this template when you need auditable, source-grounded classifications for compliance, content moderation, ticket routing, or any workflow where a label without a citation is insufficient.

textCopy
You are a classification system that assigns labels to an input based ONLY on the provided retrieved context. You must cite the exact passage that supports each label. Do not use external knowledge.

## INPUT
[INPUT]

## RETRIEVED CONTEXT
[CONTEXT]

## LABEL TAXONOMY
[LABELS]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "classifications": [
    {
      "label": "string (must be from the taxonomy)",
      "confidence": number (0.0 to 1.0),
      "supporting_evidence": [
        {
          "passage_id": "string (from context)",
          "quoted_text": "string (exact quote from passage)",
          "rationale": "string (why this evidence supports the label)"
        }
      ],
      "counter_evidence": [
        {
          "passage_id": "string",
          "quoted_text": "string",
          "rationale": "string (why this evidence weakens or contradicts the label)"
        }
      ]
    }
  ],
  "unclassifiable": boolean,
  "unclassifiable_reason": "string (required if unclassifiable is true)",
  "evidence_gaps": ["string (list of information types missing from context that would improve classification)"]
}

## CONSTRAINTS
[CONSTRAINTS]

## EXAMPLES
[EXAMPLES]

## INSTRUCTIONS
1. Read the INPUT and all RETRIEVED CONTEXT passages.
2. For each applicable label in the LABEL TAXONOMY, determine if the context supports it.
3. For every label you assign, include at least one supporting_evidence entry with an exact quote.
4. If evidence contradicts a label you considered, include it in counter_evidence.
5. Set confidence based on evidence strength, not label frequency. A single definitive passage can justify high confidence.
6. If no label can be reliably assigned from the context, set unclassifiable to true and explain why.
7. List any evidence_gaps that prevented confident classification.
8. Do not invent labels. Use only labels from the LABEL TAXONOMY.
9. Do not fabricate quotes. Every quoted_text must appear verbatim in the provided context.

Adapt this template by replacing the square-bracket placeholders at runtime. [INPUT] receives the text to classify. [CONTEXT] receives the formatted retrieved passages, each with a unique passage_id. [LABELS] receives your taxonomy as a structured list with descriptions. [CONSTRAINTS] can inject additional rules such as minimum confidence thresholds, required evidence counts, or domain-specific exclusion criteria. [EXAMPLES] should contain one or two few-shot demonstrations showing correct classification with evidence citations. After substituting variables, validate that the assembled prompt does not exceed your model's context window and that all passage_ids in [CONTEXT] are unique and traceable back to your retrieval system.

This prompt is not a standalone artifact; it is a component inside a larger RAG pipeline. The application layer is responsible for retrieving candidate passages, assembling the prompt with the correct evidence, instructions, and input, and then validating the model's output before it reaches a downstream system or user. The harness must treat the prompt as a deterministic function of its inputs: [INPUT_TEXT], [RETRIEVED_EVIDENCE], [LABEL_SET], and [CONFIDENCE_THRESHOLD]. Before calling the model, validate that retrieved passages are deduplicated, within the token budget, and formatted with source identifiers that the prompt expects. A preflight check should confirm that the assembled prompt does not exceed the model's context limit and that all required placeholders are populated with non-empty values.

After receiving the model's classification output, run a structured validation layer. Parse the JSON response and confirm that every predicted label exists in the provided [LABEL_SET], that each label's supporting evidence citations reference valid passage IDs from the retrieval set, and that confidence scores fall within 0.0 to 1.0. If the output fails schema validation, retry once with the same prompt plus a repair instruction that includes the specific validation error. If the retry also fails, log the failure, fall back to a lower-confidence threshold, or escalate for human review depending on the risk level configured in [RISK_LEVEL]. For high-stakes classification tasks, always route outputs with confidence scores below [CONFIDENCE_THRESHOLD] or with evidence-class misalignment to a human review queue.

Instrument the harness with structured logs that capture the retrieval query, the number of passages retrieved, the final prompt token count, the raw model output, validation results, and any retry attempts. This trace data is essential for debugging classification drift, evidence quality degradation, or prompt assembly errors. Avoid wiring this prompt directly into a synchronous user-facing flow without a timeout and a circuit breaker; model latency can spike under load, and a stuck classification should not block the entire application. For batch classification workloads, consider running consistency checks across similar inputs to detect unstable label assignments before trusting the output at scale.