Evidence Matching with Partial Match Handling Prompt

This prompt is designed for verification engineers and RAG pipeline builders who face a specific failure mode: over-confident verification from partial evidence. In production fact-checking systems, a source often confirms some components of a claim but not others—for example, a document may verify a person's role but not the specific action attributed to them, or a retrieved passage may corroborate a numerical trend but differ in magnitude. Binary verification prompts that output only 'supported' or 'unsupported' collapse this nuance, either accepting a claim on weak grounds or rejecting it despite substantial corroboration. This template solves that problem by forcing the model to decompose a claim into its constituent atomic components, evaluate each independently against the provided evidence, and produce a structured partial-match score with explicit identification of which parts are supported, which are not, and why.

Use this prompt in the evidence-matching stage of a verification workflow, after claim extraction and evidence retrieval have completed, and before final verification reporting. It expects a pre-extracted claim (ideally decomposed into sub-claims by an upstream process) and a set of retrieved evidence passages. The output is a structured JSON object containing a partial-match score, a component-by-component support assessment, and a summary of evidence gaps. This prompt is not a replacement for claim extraction or evidence retrieval—it assumes those steps have already produced clean inputs. It also should not be used when the evidence set is empty; pair it with a null-evidence handling prompt for that case. For high-stakes domains like healthcare, finance, or legal verification, always route partial-match outputs to human review rather than treating the score as a final determination.

Before integrating this prompt into your pipeline, calibrate the partial-match threshold against your domain's tolerance for uncertainty. A 70% match may be sufficient for news monitoring but unacceptable for regulatory compliance. Build eval datasets with known partial-match scenarios—claims where exactly half the components are supported, claims with near-miss numerical evidence, and claims where support is distributed across multiple sources—to validate that the model isn't defaulting to binary thinking despite the structured output format. The most common production failure is the model treating a high partial-match score as equivalent to full verification; mitigate this by always surfacing the unsupported components explicitly in downstream reporting, never just the aggregate score.

This template is designed for verification pipelines where evidence often partially supports a claim without confirming every component. It forces the model to decompose a claim into atomic sub-claims, match each against the provided evidence, and produce a structured output that distinguishes supported, unsupported, and contradicted elements. The output includes a partial-match score, a detailed component-level breakdown, and an overall verification status. This prevents the common failure mode where a model marks a claim as 'supported' because one part matches, ignoring the rest.

textCopy
You are an evidence verification system. Your task is to match a claim against provided evidence and produce a structured partial-match assessment.

INPUT:
- CLAIM: [CLAIM_TEXT]
- EVIDENCE: [EVIDENCE_TEXT]
- EVIDENCE_SOURCE: [SOURCE_IDENTIFIER]
- DOMAIN: [DOMAIN]

INSTRUCTIONS:
1. Decompose the CLAIM into atomic sub-claims. Each sub-claim must be a single verifiable assertion.
2. For each sub-claim, determine its match status against the EVIDENCE:
   - SUPPORTED: The evidence directly confirms this sub-claim.
   - UNSUPPORTED: The evidence does not address this sub-claim.
   - CONTRADICTED: The evidence directly contradicts this sub-claim.
3. For each SUPPORTED sub-claim, provide the exact quote from EVIDENCE that supports it.
4. Calculate a PARTIAL_MATCH_SCORE as: (number of SUPPORTED sub-claims) / (total sub-claims).
5. Determine OVERALL_STATUS:
   - FULLY_SUPPORTED: All sub-claims are SUPPORTED (score = 1.0).
   - PARTIALLY_SUPPORTED: Some sub-claims are SUPPORTED, none are CONTRADICTED (0 < score < 1.0).
   - PARTIALLY_CONTRADICTED: Some sub-claims are SUPPORTED, at least one is CONTRADICTED.
   - UNSUPPORTED: No sub-claims are SUPPORTED, none are CONTRADICTED (score = 0).
   - CONTRADICTED: At least one sub-claim is CONTRADICTED, none are SUPPORTED.
6. If EVIDENCE is empty or irrelevant, set OVERALL_STATUS to NO_EVIDENCE and PARTIAL_MATCH_SCORE to null.

OUTPUT_SCHEMA:
{
  "claim": "string",
  "evidence_source": "string",
  "sub_claims": [
    {
      "sub_claim_text": "string",
      "status": "SUPPORTED | UNSUPPORTED | CONTRADICTED",
      "supporting_quote": "string | null",
      "explanation": "string"
    }
  ],
  "partial_match_score": "number | null",
  "overall_status": "FULLY_SUPPORTED | PARTIALLY_SUPPORTED | PARTIALLY_CONTRADICTED | UNSUPPORTED | CONTRADICTED | NO_EVIDENCE",
  "unsupported_components_summary": "string"
}

CONSTRAINTS:
- Do not infer support beyond what the evidence explicitly states.
- If the evidence implies support but does not state it directly, mark the sub-claim as UNSUPPORTED.
- Do not fabricate quotes. If no quote exists, set supporting_quote to null.
- The unsupported_components_summary must list which parts of the claim lack evidence.

Adaptation guidance: Replace [CLAIM_TEXT] with the assertion to verify, [EVIDENCE_TEXT] with the retrieved passage or document content, [SOURCE_IDENTIFIER] with a document ID, URL, or chunk reference, and [DOMAIN] with the subject area (e.g., 'medical', 'legal', 'financial') to help the model calibrate its decomposition logic. For batch processing, wrap this template in a loop that iterates over claim-evidence pairs. For RAG pipelines, populate [EVIDENCE_TEXT] with the top-k retrieved passages and run this prompt once per passage, then aggregate results with a separate scoring step. If your evidence source contains multiple documents, consider using the Multi-Source Evidence Matching Prompt instead.

Validation and testing: Before deploying, validate outputs against a golden dataset of 50+ claim-evidence pairs with known partial-match outcomes. Check that partial_match_score is mathematically consistent with the sub-claim counts, that supporting_quote fields contain verbatim text from the evidence when present, and that UNSUPPORTED sub-claims never have a non-null quote. Common failure modes include the model marking a sub-claim as SUPPORTED based on inference rather than explicit evidence, and the model failing to decompose compound claims into atomic units. Add a post-processing validator that rejects outputs where supporting_quote text is not a substring of the input evidence. For high-stakes domains, route all PARTIALLY_SUPPORTED and PARTIALLY_CONTRADICTED outputs to human review with the full sub-claim breakdown attached.

The partial match prompt is not a standalone tool; it is a scoring and decomposition step inside a larger evidence-matching pipeline. In production, you will typically call this prompt after a retrieval step has returned candidate evidence passages and a claim extraction step has produced atomic claims. The prompt's job is to produce a structured partial-match assessment for each claim-evidence pair, identifying which components of the claim are supported, which are not, and why. This output feeds downstream decisions: auto-verification for high-confidence full matches, human review for partial matches above a threshold, and 'unverified' labeling for low-confidence or no-evidence cases.

Wire the prompt into your application with a strict JSON schema validator on the output. The expected output shape should include fields such as claim_id, evidence_id, overall_match_score (a float from 0.0 to 1.0), supported_components (an array of claim fragments with evidence quotes), unsupported_components (an array of claim fragments with explicit 'no supporting evidence found' notes), and match_explanation (a short string). Use a JSON Schema or Pydantic model to validate this structure before the output enters any database or routing logic. If validation fails, retry once with the same prompt and a repair instruction appended: 'The previous output did not match the required schema. Please correct and return valid JSON only.' If the second attempt fails, log the failure and route the claim-evidence pair to a human review queue with the raw model output attached.

Model choice matters here. Partial match decomposition requires strong instruction-following and structured output discipline. Prefer models with native JSON mode or constrained decoding (e.g., GPT-4o with response_format, Claude 3.5 Sonnet with tool-use-as-structured-output, or fine-tuned open-weight models with grammar-constrained generation). Avoid models that tend to produce narrative explanations instead of structured decompositions. Set temperature low (0.0–0.2) to reduce variance in match scores and component identification. For high-throughput pipelines, batch multiple claim-evidence pairs into a single prompt call with an array output schema, but cap batch size at 5–10 pairs to prevent the model from losing precision on individual assessments.

Implement a scoring threshold router after the prompt returns. Define two thresholds: AUTO_VERIFY_THRESHOLD (e.g., 0.9) and HUMAN_REVIEW_THRESHOLD (e.g., 0.5). Claims with overall_match_score >= AUTO_VERIFY_THRESHOLD and zero unsupported components can be auto-verified. Claims with overall_match_score >= HUMAN_REVIEW_THRESHOLD but with unsupported components should be routed to a human review queue with the full partial-match output as context. Claims below HUMAN_REVIEW_THRESHOLD should be labeled 'unverified' and flagged for evidence gap analysis. Log every routing decision with the claim ID, evidence ID, score, and threshold values for auditability.

Common failure modes to monitor in production: (1) the model assigns high match scores to claims where the evidence supports only a trivial or tangential component while missing the core assertion; (2) the model splits a single atomic claim into too many granular components, inflating the supported ratio; (3) the model fails to identify when evidence partially contradicts a claim, treating contradiction as 'unsupported' instead of flagging it explicitly. Mitigate these by running periodic eval passes with a golden dataset of human-annotated partial matches, and by adding a contradiction-detection field to the output schema (contradicted_components) if your domain requires it. For high-risk domains such as healthcare or legal verification, always require human review for any partial match, regardless of score, and never auto-verify claims with any unsupported components.