Misinformation and Disinformation Risk Scoring Prompt

This prompt is for integrity and safety engineers who need a structured, auditable risk signal to control content distribution, recommendation system inputs, or model training data. The job-to-be-done is triage: you have a high volume of content that may contain false or misleading claims, and you need to decide what gets blocked, labeled, demoted, or escalated for human review before it causes harm. The ideal user is an engineer or operator embedding this prompt into a production pipeline where every decision must be traceable to a claim, a checkability assessment, and a harm evaluation. You should use this prompt when your platform needs a consistent, repeatable scoring layer that sits between content ingestion and distribution—not as a replacement for fact-checking organizations, but as a routing and prioritization tool that reduces the surface area of un-reviewed misinformation.

The prompt requires several concrete inputs to function reliably. You must provide the content to be scored as [INPUT], a [CONTEXT] block that defines your platform's risk tolerance and content policies, and an [OUTPUT_SCHEMA] that specifies the exact JSON structure you expect (including claim extraction fields, checkability scores, harm assessments, risk tiers, and recommended actions). You should also supply a [FACT_CHECK_CORPUS] or retrieval mechanism that grounds the model's checkability assessment in known fact-check records—without this grounding, the prompt will produce plausible-sounding but unverifiable confidence scores. The prompt works best when paired with a validation layer that checks output structure, enforces enum values for risk tiers, and flags outputs where the model claims high confidence without citing specific fact-check sources. For high-stakes domains like public health, election integrity, or crisis response, you must route all outputs through a human review queue regardless of the model's confidence score.

Do not use this prompt when you need a definitive truth judgment, when the content requires real-time fact-checking against breaking news, or when the cost of a false negative is catastrophic without human review. This prompt is a triage and routing tool, not a publisher. It will fail on satire, opinion pieces, and highly contextual claims where checkability is inherently low. Before deploying, calibrate the risk tiers against a labeled dataset that reflects your platform's actual content distribution, and run regular evals to detect drift in the model's sensitivity to new misinformation narratives. The next step after reading this section is to copy the prompt template, adapt the placeholders to your taxonomy, and build the validation harness that will catch malformed outputs before they reach your distribution controls.

This prompt is designed to be dropped into your AI safety pipeline. It instructs the model to act as an integrity analyst, extracting factual claims from the input, assessing their checkability, and evaluating potential harm. The output is a structured JSON object suitable for automated routing, logging, and human review queues. Before integrating, ensure your system can handle the required input variables and that you have a plan for evaluating the output against a ground-truth dataset.

textCopy
You are an integrity and safety analyst. Your task is to score the provided content for misinformation and disinformation risk. You must extract verifiable claims, assess their checkability, and evaluate the potential for real-world harm.

Follow these steps:
1.  **Claim Extraction:** Identify and list all discrete, factual claims made in the content. Ignore opinions, questions, and rhetorical statements.
2.  **Checkability Assessment:** For each claim, determine if it is verifiable using authoritative, publicly available sources. Classify each as 'Checkable' or 'Uncheckable'.
3.  **Harm Potential Evaluation:** Analyze the potential for the content, if false or misleading, to cause harm in categories such as public health, public safety, democratic processes, or individual reputation. Rate the overall harm potential as 'Low', 'Medium', 'High', or 'Critical'.
4.  **Risk Scoring:** Synthesize the checkability and harm assessments to assign an overall risk tier: 'Tier 1 (Low Risk)', 'Tier 2 (Medium Risk)', 'Tier 3 (High Risk)', or 'Tier 4 (Critical Risk)'.
5.  **Recommend Action:** Based on the risk tier, recommend a single action: 'No Action', 'Label Content', 'Reduce Distribution', or 'Escalate for Urgent Review'.

## INPUT
[CONTENT_TO_SCORE]

## CONSTRAINTS
- Base your assessment strictly on the provided content. Do not introduce external knowledge.
- If no factual claims are present, the risk tier must be 'Tier 1 (Low Risk)' and the action 'No Action'.
- For any risk tier of 'Tier 3' or above, the recommended action must be 'Reduce Distribution' or 'Escalate for Urgent Review'.
- Your output must be a single, valid JSON object matching the output schema. Do not include any other text.

## OUTPUT_SCHEMA
{
  "claims": [
    {
      "claim_text": "string",
      "checkability": "Checkable | Uncheckable"
    }
  ],
  "overall_harm_potential": "Low | Medium | High | Critical",
  "risk_tier": "Tier 1 (Low Risk) | Tier 2 (Medium Risk) | Tier 3 (High Risk) | Tier 4 (Critical Risk)",
  "recommended_action": "No Action | Label Content | Reduce Distribution | Escalate for Urgent Review",
  "rationale": "A concise summary explaining the risk tier and action."
}

To adapt this template, replace the [CONTENT_TO_SCORE] placeholder with the text you need to evaluate. You can further refine the behavior by adding few-shot examples to the [EXAMPLES] section or by adjusting the harm categories in the prompt body to match your platform's specific policies. For high-stakes applications, always route outputs with a risk_tier of 'Tier 3' or above to a human review queue and log the full JSON payload for auditing.

This prompt is designed to sit behind an API endpoint or queue worker that receives content payloads—articles, social posts, transcripts—and returns a structured risk assessment. The harness must treat the model's output as a preliminary signal, not a final decision. Because misinformation scoring carries direct reputational and policy consequences, the implementation must include strict output validation, confidence thresholding, and a mandatory human-review path for high-severity or low-confidence results. The prompt expects a [CONTENT] input, an optional [CONTEXT] block with known fact-check corpora or editorial guidelines, and a defined [OUTPUT_SCHEMA] that includes risk tier, extracted claims, checkability scores, and recommended actions. Wire this prompt into a pipeline where the input is sanitized and truncated to a maximum token length before invocation to prevent context-window overruns that silently drop claims from analysis.

Start with a validation layer that parses the model's JSON response and checks for schema compliance before any downstream action. Required fields include risk_tier (must be one of LOW, MEDIUM, HIGH, CRITICAL), claims (array of objects with claim_text, checkability_score, and harm_potential), and recommended_action (one of NO_ACTION, LABEL, REDUCE_DISTRIBUTION, ESCALATE). If the model returns malformed JSON, missing required fields, or enum values outside the allowed set, the harness should retry once with a repair prompt that includes the validation error message. After two consecutive failures, log the incident and route to a human review queue. For model choice, use a model with strong reasoning and structured output capabilities—GPT-4o or Claude 3.5 Sonnet work well—and set temperature=0 to maximize consistency across repeated scoring runs. Implement confidence gating: if the model's self-reported confidence score (included in the output schema) falls below 0.7, or if the risk tier is HIGH or CRITICAL, automatically escalate to human review regardless of schema validity. Log every invocation with the input hash, model version, output, validation result, and routing decision for auditability.

For grounding evaluation, maintain a holdout set of fact-checked claims from sources like the International Fact-Checking Network (IFCN) or your organization's internal editorial corpus. Run this prompt against that holdout set weekly and compare the model's risk tier assignments and claim checkability scores against ground truth. Track precision and recall for HIGH/CRITICAL classifications specifically—false negatives here mean harmful misinformation passing through unlabeled. If precision drops below 85% or recall below 90%, freeze the prompt version and investigate before updating production. Avoid wiring this prompt directly to automated content takedown or account suspension actions; the output should always feed a review queue or labeling system first. The highest-risk failure mode is over-escalation on legitimate but controversial content, which erodes trust and creates editorial bottlenecking. Monitor the ratio of ESCALATE to LABEL actions weekly and investigate if the escalation rate exceeds 10% of total classifications without a corresponding external event driving the increase.