Corroboration Count and Agreement Scoring Prompt Template

This prompt is for verification system architects and pipeline engineers who need to measure how many independent sources support a claim. It produces a corroboration score with source independence checks, agreement strength, and redundancy penalties. Use this when your verification pipeline already has extracted claims and retrieved evidence passages, and you need to distinguish true corroboration from echo citations before assigning a final confidence score. This prompt belongs after evidence retrieval and before confidence scoring or human-review routing. It is not a replacement for evidence retrieval itself, and it assumes you have already matched claims to candidate evidence sources.

The ideal user is building an automated fact-checking pipeline where a single claim has been paired with multiple evidence snippets from a retrieval step. The prompt expects a structured input containing the claim text and a list of evidence objects, each with source metadata (e.g., publisher, date, author) and the relevant passage. The output is a numerical corroboration score, an independence assessment, and a breakdown of agreement strength. Do not use this prompt for claims with zero evidence matches, for source authority scoring (use a dedicated authority prompt), or for final binary true/false classification. It is a signal component, not a verdict.

Before wiring this into production, ensure your evidence retrieval step returns multiple candidates per claim. A common failure mode is feeding this prompt a single evidence item and expecting a meaningful corroboration score—the output will be trivially 1.0 with no independence check. Also, implement a validation layer that rejects outputs where the model claims independence but the sources share the same root wire service or press release. The next step after this prompt is typically a confidence calibration prompt that consumes the corroboration score alongside authority and contradiction signals.

This prompt template is designed for verification pipelines that need to distinguish true corroboration from echo citations. It instructs the model to analyze a set of sources against a target claim, evaluate source independence, measure agreement strength, and apply redundancy penalties when sources merely repeat the same underlying evidence. The output is a structured corroboration score with a detailed breakdown that can be consumed by downstream scoring, triage, or audit systems.

codeCopy
You are a corroboration analyst evaluating how many independent sources support a given claim. Your task is to produce a corroboration score with source independence checks, agreement strength measurement, and redundancy penalties.

## INPUT

**Target Claim:**
[CLAIM_TEXT]

**Claim Domain (for authority context):**
[CLAIM_DOMAIN]

**Source List with Evidence:**
[SOURCES]
<!-- Format: For each source, provide:
- source_id: unique identifier
- source_name: publication or author name
- source_type: e.g., primary_research, news_report, government_record, expert_testimony, corporate_disclosure, social_media, unknown
- evidence_excerpt: the specific passage that supports or contradicts the claim
- publication_date: ISO date or best estimate
- retrieval_context: how this source was obtained (e.g., web_search, knowledge_base, user_provided)
-->

## OUTPUT SCHEMA

Return a JSON object with this exact structure:
{
  "corroboration_summary": {
    "total_sources_reviewed": <integer>,
    "independent_source_count": <integer>,
    "echo_source_count": <integer>,
    "corroboration_score": <float 0.0-1.0>,
    "score_label": "<high_corroboration | moderate_corroboration | low_corroboration | single_source_only | no_corroboration>"
  },
  "independence_analysis": [
    {
      "source_group_id": "<string>",
      "group_label": "<independent | echo_cluster>",
      "source_ids": ["<string>"],
      "independence_rationale": "<explain why these sources are or are not independent, citing ownership, authorship, data origin, or citation chains>"
    }
  ],
  "agreement_assessment": {
    "agreement_level": "<full_agreement | partial_agreement | neutral | partial_disagreement | full_disagreement>",
    "agreement_strength": <float 0.0-1.0>,
    "key_points_of_agreement": ["<string>"],
    "key_points_of_divergence": ["<string>"]
  },
  "redundancy_penalties": [
    {
      "source_ids": ["<string>"],
      "penalty_reason": "<explain the redundancy: e.g., same underlying dataset, syndicated content, shared author, circular citation>",
      "penalty_weight": <float 0.0-1.0>
    }
  ],
  "scoring_methodology": {
    "base_score": <float>,
    "independence_multiplier": <float>,
    "agreement_factor": <float>,
    "redundancy_deduction": <float>,
    "final_score": <float>
  },
  "limitations_and_caveats": ["<string>"]
}

## CONSTRAINTS

1. **Independence check:** Two sources are NOT independent if one cites the other as its primary evidence, they share the same author or parent organization, they syndicate the same wire content, or they reference the same single underlying dataset without additional verification. Document your reasoning.
2. **Echo detection:** If multiple sources repeat the same claim without adding new evidence, investigative work, or verification, classify them as an echo cluster and apply a redundancy penalty.
3. **Agreement measurement:** Assess whether sources agree on the core factual content of the claim, not just surface wording. Partial agreement occurs when sources support the claim but differ on specifics like magnitude, timeline, or scope.
4. **Score calibration:** A corroboration score of 1.0 requires multiple fully independent sources in full agreement with no redundancy. A score of 0.0 means no sources support the claim or all sources are echoes of a single origin.
5. **Single-source handling:** If only one independent source exists, set `corroboration_score` to 0.3 maximum and label as `single_source_only`, regardless of source authority.
6. **Unknown source type:** If source_type is `unknown`, treat the source as lower weight but still analyze its content. Note the uncertainty in `limitations_and_caveats`.
7. **Missing evidence:** If a provided source does not actually contain evidence about the claim, exclude it from scoring and note it in `limitations_and_caveats`.
8. **Domain context:** Use [CLAIM_DOMAIN] to assess what counts as independent verification in that field (e.g., peer-reviewed replication in science, multiple eyewitness accounts in news, separate audit trails in finance).

## EXAMPLES

**Example 1: High Corroboration**
Claim: "Global average temperature increased by approximately 1.1°C since pre-industrial levels."
Sources: IPCC report (primary_research), NASA GISS dataset (primary_research), NOAA annual climate report (government_record)
Expected: independent_source_count=3, echo_source_count=0, corroboration_score>=0.9, score_label="high_corroboration"

**Example 2: Echo Cluster**
Claim: "Company X announced a new product launch next quarter."
Sources: Press release on Company X website (corporate_disclosure), TechCrunch article citing the press release (news_report), Reuters wire story (news_report)
Expected: independent_source_count=1, echo_source_count=2, corroboration_score<=0.4, redundancy penalties applied

**Example 3: Single Source Only**
Claim: "A new species of deep-sea fish was discovered in the Mariana Trench."
Sources: Single research paper (primary_research)
Expected: independent_source_count=1, echo_source_count=0, corroboration_score<=0.3, score_label="single_source_only"

## RISK LEVEL
[RISK_LEVEL]
<!-- Use "high" for regulated domains, safety-critical claims, or public-facing verification. Use "standard" for internal analysis. High risk requires more conservative scoring and explicit uncertainty flags. -->

To adapt this template for your pipeline, replace the bracketed placeholders with your actual claim text, domain label, and structured source list. The [SOURCES] placeholder expects a formatted list—consider building a pre-processing step that normalizes source metadata before insertion. For high-risk domains, set [RISK_LEVEL] to high to trigger more conservative scoring behavior. After receiving the model output, validate the JSON structure against the schema, verify that final_score matches the scoring methodology calculation, and log any limitations_and_caveats for downstream review. Run this prompt against a golden set of claims with known corroboration patterns before deploying to production.

The Corroboration Count and Agreement Scoring prompt is designed to sit downstream of claim extraction and evidence retrieval. In a production pipeline, you will typically call this prompt once per claim, passing the claim text and a structured list of retrieved evidence items. The model returns a JSON object containing a corroboration score, source independence assessment, agreement strength, and redundancy penalties. This output should be treated as a structured signal that feeds into a broader confidence scoring or triage system—not as a final verdict on its own.

Input assembly: Before calling the prompt, normalize your evidence payload. Each evidence item should include at minimum a source_id, source_name, content_snippet, and retrieval_timestamp. If you have source authority metadata (domain, publication type, author credentials), include it as optional fields. The prompt template uses the [CLAIM] and [EVIDENCE_ITEMS] placeholders—populate [EVIDENCE_ITEMS] as a JSON array string. Validation: Wrap the model call in a JSON schema validator that checks for required output fields: corroboration_count, independent_source_count, agreement_strength (0.0–1.0), redundancy_penalty_applied (boolean), and source_independence_notes (array of strings). If validation fails, retry once with the error message appended to the prompt context. If the second attempt fails, route the claim to a human review queue with the raw model output attached.

Model choice and latency: This prompt works well with models that have strong instruction-following and structured output capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). For high-throughput pipelines processing thousands of claims, consider batching claims with shared evidence sets to reduce redundant context. Logging: Log every corroboration score alongside the claim ID, evidence IDs, model version, and timestamp. This audit trail is essential for debugging score drift when you update the prompt or switch models. Routing: Use the agreement_strength and independent_source_count fields as inputs to your triage threshold logic. For example, claims with agreement_strength < 0.3 or independent_source_count < 2 might automatically route to human review, while claims above 0.7 with 3+ independent sources could auto-verify. What to avoid: Do not use this prompt on claims where the evidence set is empty or contains only a single source—the output will be degenerate. Always pre-filter claims with a minimum evidence threshold before invoking corroboration scoring. Also avoid treating high corroboration as truth; independent sources can all repeat the same error. Pair this prompt with source authority and contradiction detection steps for a complete verification signal.