Evidence Score Justification Prompt with Rubric

This prompt is for compliance engineers, AI quality teams, and RAG pipeline architects who need auditable, rubric-based justifications for every evidence score assigned to a retrieved passage. The job-to-be-done is not just scoring evidence, but producing a traceable artifact that explains why a specific weight was chosen, linking each factor in the rubric directly to content in the source passage. Use this when downstream decisions—such as answer generation, regulatory filings, or internal audit reviews—depend on being able to replay and defend the evidence selection logic. The ideal user has a retrieval set in hand and needs to convert raw passages into a scored, justified, and reviewer-ready evidence ledger.

Do not use this prompt for real-time, low-latency re-ranking where a simple numerical score without justification is sufficient. It is overkill for applications that only need a relevance sort order. Avoid it when the evidence passages are extremely short (e.g., single-sentence fragments) and lack enough content to support multi-factor rubric analysis. This prompt is also a poor fit when the retrieval set is uncurated web-scale noise; it assumes passages have already passed a basic relevance filter. If you need a lightweight relevance score, use the 'Query-to-Passage Relevance Scoring Prompt' instead. If you need a simple weighted score without detailed justifications, start with the 'Multi-Factor Evidence Weighting Prompt Template'.

Before using this prompt, ensure you have defined your scoring rubric with explicit factors (e.g., relevance, recency, authority, specificity) and weight ranges. The prompt template expects you to supply this rubric as part of the context. After generation, every output must pass a validation check: each justification must contain a direct quote or specific reference to the passage content. If a justification is vague or untethered from the source, the output fails. In high-stakes domains, plan for human review of a sample of scored passages to calibrate the rubric and detect systematic scoring drift. The next step is to wire this prompt into a pipeline that logs the full input, output, and reviewer decisions for audit trail generation.

This template is designed to be dropped directly into your evidence scoring pipeline. It instructs the model to act as an evidence auditor, scoring a single passage against a configurable rubric of factors—relevance, recency, authority, specificity, and query alignment—and then justifying each score with direct quotes from the passage. The output is a structured JSON object suitable for logging, downstream filtering, and compliance review. The square-bracket placeholders allow you to inject your specific query, passage, rubric weights, and output constraints without modifying the core instruction logic.

textCopy
You are an evidence auditor. Your task is to score a single evidence passage against a rubric and produce a detailed justification for each score. Your output must be auditable, traceable, and grounded in the passage content.

## INPUTS
- Query: [QUERY]
- Passage: [PASSAGE]
- Rubric Factors: [RUBRIC_FACTORS]
  Format: A list of factors, each with a name, description, and weight (0.0 to 1.0).
- Scoring Scale: [SCORING_SCALE]
  Format: A description of the scale, e.g., "1-5, where 1 is lowest and 5 is highest."
- Output Schema: [OUTPUT_SCHEMA]
  Format: A valid JSON schema that the output must conform to.
- Constraints: [CONSTRAINTS]
  Format: A list of rules the justification must follow, e.g., "Every justification must include a direct quote from the passage."
- Risk Level: [RISK_LEVEL]
  Options: "low", "medium", "high". If "high", you must flag any score below 3 for human review.

## INSTRUCTIONS
1. For each factor in [RUBRIC_FACTORS], assign a score based on [SCORING_SCALE].
2. Write a justification for each score. The justification must:
   - Reference specific content from [PASSAGE] using direct quotes.
   - Explain how the passage content maps to the factor's description.
   - Note any missing information that prevented a higher score.
3. Calculate a weighted total score by multiplying each factor score by its weight and summing the results.
4. If [RISK_LEVEL] is "high", add a `human_review_required` boolean field. Set it to `true` if any individual factor score is below 3.
5. Output ONLY a valid JSON object that strictly conforms to [OUTPUT_SCHEMA]. Do not include any text outside the JSON object.

## OUTPUT

Adapting the template: Replace each square-bracket placeholder with concrete values at runtime. For [RUBRIC_FACTORS], provide a structured list—typically injected as a serialized JSON array or a formatted markdown table. For [OUTPUT_SCHEMA], supply a strict JSON Schema definition that your application's parser expects; this is your primary defense against malformed outputs. The [CONSTRAINTS] field is where you enforce domain-specific rules, such as 'justifications must not exceed 150 words' or 'scores of 1 must explain why the passage is unusable.' For high-stakes compliance workflows, set [RISK_LEVEL] to "high" and ensure your application layer respects the human_review_required flag by routing flagged outputs to a review queue. Always validate the model's output against [OUTPUT_SCHEMA] before accepting it into your pipeline; if validation fails, retry with the error message appended to [CONSTRAINTS].

Next step: After copying this template, define your rubric factors and their weights carefully. The quality of the justification depends on how well the factor descriptions guide the model. Test with a small set of passages where you know the expected scores, and compare the model's justifications against your own reasoning. If the model consistently overweights a factor like recency when authority should dominate, adjust the factor weights or add explicit counter-instructions to [CONSTRAINTS]. For production, log every scored output—including the full prompt, passage, and model response—to create an audit trail that supports debugging and compliance review.

The Evidence Score Justification Prompt is designed to sit between your retrieval step and your answer generation or evidence selection step. In a typical RAG pipeline, you retrieve N candidate passages, then call this prompt for each passage (or batch them if your context window allows) to produce a structured justification object. The output is not just a score—it's a rubric-anchored explanation that links every factor weight to specific passage content. This makes the prompt suitable for compliance workflows where downstream reviewers need to understand why a passage was ranked above another, not just that it was.

To wire this into an application, wrap the prompt call in a function that accepts a query string, a passage object (with text, source metadata, and publication date), and a rubric configuration. The function should construct the prompt by injecting these into the [QUERY], [PASSAGE_TEXT], [PASSAGE_METADATA], and [RUBRIC] placeholders. On the output side, parse the JSON response and validate it against a strict schema: require score (number 0-1), factor_breakdown (object with required keys matching your rubric dimensions), justification (string with explicit passage references), and traceability_checks (array of objects linking each justification claim to a passage span). If validation fails, retry once with the validation error injected into [CONSTRAINTS]. If it fails again, route the passage to a human review queue rather than silently accepting a malformed score.

For model choice, use a model with strong instruction-following and JSON output capabilities—GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro are appropriate. Avoid smaller models for this task because the structured justification requires precise reasoning about rubric criteria and passage content simultaneously. If you're processing high volumes, consider batching up to 5 passages per call with an array output schema, but monitor for attention degradation across passages. Log every scored passage with its raw prompt, raw response, parsed score, and validation status to a scoring audit table. This log becomes your evidence trail when a compliance reviewer asks why a particular passage was deprioritized six months ago. Do not use this prompt for real-time user-facing latency budgets under 500ms—the justification generation adds meaningful inference time.

The most common production failure mode is rubric-passage misalignment: the model justifies a score using rubric criteria that don't actually appear in the passage, producing a plausible-sounding but unfaithful justification. Your traceability_checks field is the primary defense. In your post-processing step, verify that each traceability entry's passage_span actually exists in the input passage text. If spans don't match, flag the scoring result as unreliable and escalate. A secondary failure mode is score inflation on ambiguous passages—the model assigns moderate-to-high scores to passages that sound relevant but lack concrete support. Mitigate this by including a specificity dimension in your rubric and calibrating your score thresholds against a human-labeled validation set before deployment. Run periodic eval batches comparing model-assigned scores against expert judgments, and trigger a rubric review if drift exceeds 0.15 mean absolute error.