Evidence-to-Claim Alignment Scoring Prompt

This prompt is designed for evaluation engineers and fact-checking pipeline builders who need a calibrated, repeatable method for scoring how well a retrieved passage supports or refutes a specific claim. The primary job-to-be-done is automated evidence quality assessment: given a claim and a candidate passage, produce an alignment score on a defined scale along with reasoning that explains the score. This is a critical component in RAG evaluation harnesses, claim verification systems, and any production pipeline where grounding fidelity must be measured programmatically before answers reach users.

Use this prompt when you have already retrieved candidate passages and need to rank or filter them by evidentiary strength. It is appropriate for offline evaluation runs, online guardrails that check answer grounding before display, and batch scoring of evidence-claim pairs for dataset curation. The prompt expects structured inputs: a single claim and a single passage. It is not designed for multi-hop reasoning across documents, open-ended summarization, or direct answer generation. The output is a score and reasoning, not a final answer. For high-stakes domains such as legal, medical, or financial compliance, always route outputs through human review and maintain audit logs of scored pairs.

Do not use this prompt when you need to compare multiple passages simultaneously, generate a grounded answer, or detect hallucinations in already-written text. Those tasks require separate playbooks for evidence ranking, RAG answer generation, or hallucination detection. This prompt is also not a substitute for human judgment in safety-critical decisions; it is a signal that should feed into a larger evaluation or guardrail system. Before deploying, calibrate the scoring rubric against human-annotated alignment judgments and establish pass/fail thresholds that match your application's tolerance for grounding errors.

The prompt below is designed for evaluation pipelines, fact-checking harnesses, and RAG quality measurement. It takes a claim and one or more evidence passages and returns an alignment score on a calibrated scale, along with reasoning that explains the score. The template uses square-bracket placeholders so you can swap in your own claim, evidence, scoring rubric, and output schema without rewriting the instruction structure.

textCopy
You are an evidence alignment scorer. Your job is to evaluate how well a retrieved passage supports or refutes a specific claim.

## INPUT
Claim: [CLAIM]
Evidence Passage: [EVIDENCE_PASSAGE]

## SCORING RUBRIC
Use the following 5-point scale to rate alignment between the claim and the evidence passage:
- 5: Directly supports the claim with explicit, unambiguous evidence.
- 4: Strongly supports the claim but requires minor inference.
- 3: Partially supports the claim; some elements align, others are missing or tangential.
- 2: Weakly related; the passage mentions the topic but does not substantiate the claim.
- 1: Contradicts or refutes the claim.
- 0: No relationship; the passage is irrelevant to the claim.

## CONSTRAINTS
[CONSTRAINTS]

## OUTPUT SCHEMA
Return a single JSON object with the following fields:
{
  "alignment_score": <integer 0-5>,
  "alignment_label": <"direct_support" | "strong_support" | "partial_support" | "weak_relation" | "contradicts" | "no_relationship">,
  "reasoning": <string explaining the score with specific reference to the claim and passage text>,
  "key_evidence_span": <string quoting the most relevant sentence or phrase from the passage, or null if none>,
  "missing_elements": <array of strings listing claim components not addressed by the passage>
}

## EXAMPLES
[EXAMPLES]

## INSTRUCTIONS
1. Read the claim and evidence passage carefully.
2. Identify which parts of the claim are addressed, supported, contradicted, or unmentioned.
3. Assign the alignment score that best matches the rubric.
4. Provide reasoning that cites specific text from both the claim and the passage.
5. If the passage contradicts the claim, explain the contradiction explicitly.
6. If the passage is irrelevant, set the score to 0 and explain why.
7. Return only the JSON object. Do not include any text outside the JSON.

To adapt this template, replace [CLAIM] and [EVIDENCE_PASSAGE] with your actual inputs. Populate [CONSTRAINTS] with domain-specific rules such as "ignore passages shorter than 20 words" or "treat hedging language as weak support." Add [EXAMPLES] as few-shot demonstrations showing scored claim-passage pairs, especially edge cases where partial support and contradiction are easy to confuse. For high-stakes domains like legal or clinical review, add a [RISK_LEVEL] field and require human review when the alignment score is 2 or below. Validate the output JSON against the schema before accepting the score, and log any parse failures for retry or escalation.

This prompt is designed to operate as a scoring microservice within a larger evaluation harness. It expects a single claim and a single retrieved passage as input and returns a structured alignment score with reasoning. The primary integration pattern is a batch evaluation loop: for each claim-evidence pair in your test set, call the LLM with this prompt, parse the structured output, and aggregate scores to measure retrieval precision or grounding quality. Do not use this prompt in a real-time user-facing chat flow without caching or async processing—it is an evaluation tool, not a conversational interface.

Wiring the prompt into an application requires a thin orchestration layer. In Python, wrap the prompt template as a function that accepts claim and evidence strings, injects them into the [CLAIM] and [EVIDENCE] placeholders, and calls your model endpoint. Enforce the output schema with a validator: parse the JSON response, confirm the alignment_score field is an integer between 1 and 5, check that reasoning is a non-empty string, and verify verdict is one of the allowed enum values (SUPPORTS, REFUTES, NEUTRAL, INSUFFICIENT). If validation fails, retry once with the error message appended to the prompt. Log every raw response, parsed score, and validation outcome for later trace analysis. For high-stakes pipelines (legal, clinical, financial), route INSUFFICIENT verdicts and low-confidence scores to a human review queue before they influence downstream decisions.

Model choice matters. This prompt relies on calibrated reasoning and rubric adherence. Use a model with strong instruction-following and structured output capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid smaller or older models that may drift toward always assigning a middling score or misinterpreting the REFUTES category. For cost-sensitive batch evaluation, consider using a faster model for the initial pass and escalating ambiguous cases (scores of 2-3, or NEUTRAL verdicts) to a stronger model for re-scoring. Tool use and RAG are not required for this prompt—it operates on pre-retrieved evidence. If your pipeline retrieves multiple passages per claim, call this prompt once per passage, then aggregate scores with a downstream ranking or max-score selection step. Avoid feeding multiple passages into a single prompt call; the rubric is calibrated for pairwise claim-evidence alignment, and multi-passage inputs degrade score reliability.

Testing and evaluation should include a golden dataset of claim-evidence pairs with human-annotated alignment scores. Measure agreement between model-assigned scores and human labels using quadratic weighted kappa or exact-match accuracy. Pay special attention to failure modes: the model may assign SUPPORTS to tangentially related passages, conflate REFUTES with NEUTRAL, or overuse INSUFFICIENT when the passage is short. Build a regression test suite that includes these edge cases and run it after any prompt or model version change. For production monitoring, track score distribution drift over time—a sudden shift toward higher or lower scores may indicate retrieval pipeline degradation or model behavior changes.