Misinformation and Disinformation Detection Prompt

This prompt is a detection and escalation tool for integrity teams and trust and safety engineers who need to monitor AI-generated outputs for factual inaccuracies that carry potential real-world harm. It is designed to sit inside a guardrail pipeline, not as a standalone fact-checker. Use it when an AI system produces public-facing content, answers questions about high-stakes topics such as health, elections, or public safety, or generates summaries that could be redistributed. The prompt extracts discrete claims, searches for counter-evidence, assesses virality risk, and produces a structured escalation record for human review. The core job-to-be-done is to catch and package potentially harmful misinformation before it reaches an end user, providing a human reviewer with everything they need to make a rapid adjudication decision.

The ideal user is an engineer integrating this prompt into a post-generation validation step within a larger AI pipeline. Required context includes the full AI-generated text, the user prompt that triggered it, and any source documents used for grounding. The prompt is most effective when paired with a retrieval tool that can search a fact-checking database or the open web for counter-evidence. Do not use this prompt for real-time content moderation where latency is critical, or as a replacement for a dedicated fact-checking database. It is a detection and escalation tool, not a source of truth. It should never be the final arbiter of factuality; a human reviewer must always confirm the escalation before any action is taken, especially for high-severity claims.

Before deploying, you must calibrate the prompt against a golden dataset of known misinformation examples and benign content to establish a baseline for your specific use case. Pay close attention to the false positive rate, as over-escalation will quickly fatigue human reviewers and undermine trust in the system. The prompt's effectiveness is directly tied to the quality and freshness of the evidence sources it can access. If your retrieval system cannot find counter-evidence for a novel breaking-news event, the prompt may incorrectly flag accurate reporting as misinformation. Start by running this prompt in a shadow mode alongside your existing moderation processes, comparing its escalations to human judgments for at least two full review cycles before letting it trigger any automated action or user-facing warning.

This prompt template is the core detection surface for your misinformation guardrail. It accepts content that has already been generated or submitted, extracts factual claims, evaluates them against known evidence, and produces a structured flag when harmful inaccuracies are detected. The template is designed to be wired into your post-generation validation pipeline—after content creation but before publication or user delivery—so that flagged content can be blocked, revised, or escalated to human review before it causes real-world harm.

textCopy
You are a misinformation detection system operating within a content safety pipeline. Your job is to analyze the provided content for factual inaccuracies that could cause real-world harm if disseminated. You do not moderate opinions, satire, or clearly labeled fiction. You focus exclusively on statements presented as factual that are verifiably false or misleading in ways that could harm individuals, public health, democratic processes, or public safety.

## INPUT
[CONTENT]

## CONTEXT
- Content source: [SOURCE_TYPE]
- Intended audience: [AUDIENCE]
- Publication channel: [CHANNEL]
- Jurisdiction: [JURISDICTION]

## INSTRUCTIONS
1. Extract every discrete factual claim from the content. A factual claim is a statement that can be verified or falsified with evidence. Ignore opinions, subjective statements, and clearly labeled speculation.
2. For each claim, determine whether it is:
   - **Supported**: Consistent with authoritative evidence sources.
   - **Unsupported**: No reliable evidence found to confirm or refute.
   - **False**: Contradicted by authoritative evidence.
   - **Misleading**: Technically true but presented in a way that creates a false impression likely to cause harm.
3. For claims marked False or Misleading, provide:
   - The specific claim text.
   - A concise counter-evidence statement citing authoritative sources.
   - A harm category from the list below.
4. Assess the overall content for virality risk based on emotional manipulation, outrage triggers, and shareability mechanics.
5. Produce a final severity classification.

## HARM CATEGORIES
- PUBLIC_HEALTH: False claims about medical treatments, vaccines, diseases, or health guidance.
- DEMOCRATIC_PROCESS: False claims about elections, voting procedures, candidates, or civic institutions.
- PUBLIC_SAFETY: False claims about emergencies, disasters, threats, or safety instructions.
- FINANCIAL_HARM: False claims designed to manipulate markets, defraud, or cause economic damage.
- HATE_AND_VIOLENCE: False claims that incite violence or hatred against protected groups.
- MANIPULATED_MEDIA: Deceptively altered images, audio, or video presented as authentic.
- OTHER: Factual inaccuracies with potential for harm not covered above.

## SEVERITY LEVELS
- CRITICAL: Imminent threat to life, public safety, or democratic process integrity.
- HIGH: Significant potential for widespread harm or targeted harm to vulnerable groups.
- MEDIUM: Potentially misleading with moderate harm risk.
- LOW: Minor inaccuracies with limited harm potential.

## OUTPUT SCHEMA
Return a valid JSON object with this exact structure:
{
  "detection_id": "string",
  "timestamp": "ISO-8601",
  "content_hash": "string",
  "overall_assessment": {
    "contains_harmful_misinformation": boolean,
    "severity": "CRITICAL|HIGH|MEDIUM|LOW|NONE",
    "confidence_score": 0.0-1.0,
    "summary": "One-sentence summary of findings"
  },
  "claims": [
    {
      "claim_id": "string",
      "claim_text": "Exact text of the claim",
      "verification_status": "SUPPORTED|UNSUPPORTED|FALSE|MISLEADING",
      "harm_category": "PUBLIC_HEALTH|DEMOCRATIC_PROCESS|PUBLIC_SAFETY|FINANCIAL_HARM|HATE_AND_VIOLENCE|MANIPULATED_MEDIA|OTHER|null",
      "counter_evidence": "Concise factual correction with source reference",
      "source_citation": "Authoritative source name and retrieval identifier",
      "confidence": 0.0-1.0
    }
  ],
  "virality_risk": {
    "score": 0.0-1.0,
    "factors": ["List of virality indicators present"],
    "assessment": "Brief explanation of virality risk"
  },
  "escalation_recommended": boolean,
  "escalation_reason": "string or null"
}

## CONSTRAINTS
- Do not flag statements that are clearly opinion, satire, parody, or fiction labeled as such.
- Do not flag statements about topics where scientific consensus is genuinely evolving unless the claim contradicts current authoritative guidance.
- When confidence is below 0.7 on any False or Misleading claim, mark it as UNSUPPORTED instead.
- If no harmful misinformation is detected, return overall_assessment.severity as "NONE" and an empty claims array.
- Always include source citations for counter-evidence. Never assert a claim is false without citing an authoritative source.
- If the content is in a language you cannot reliably evaluate, set confidence_score to 0.0 and escalate.

Adaptation guidance: Replace the bracketed placeholders with values from your application context. [CONTENT] should receive the full text to analyze—truncate only if token limits require it, and log when truncation occurs. [SOURCE_TYPE] should indicate origin (e.g., user_generated, model_output, third_party_syndication) to help downstream reviewers assess context. [JURISDICTION] is critical for regulatory compliance: misinformation standards differ across legal frameworks, and your harm categories may need jurisdiction-specific subcategories. Before deploying, calibrate the severity levels against your organization's escalation policy and test the output schema against your review queue ingestion format to ensure field compatibility.

This prompt is designed to sit inside a post-generation safety layer, not as a real-time user-facing chatbot. The typical integration pattern is: a model generates content (a draft, a summary, an answer), and that output is immediately passed to this detection prompt before it reaches the end user or is stored. The detection prompt returns a structured verdict with claim extraction, counter-evidence, and a virality risk assessment. Your application code then reads that verdict and decides whether to block, flag for review, or release the content.

Input assembly requires three pieces of context: the [CONTENT_TO_CHECK] (the AI-generated or user-submitted text), the [SOURCE_GROUNDING] (the retrieved evidence or knowledge base passages the original model was supposed to use), and the [AUDIENCE_AND_PLATFORM_CONTEXT] (where this content will appear and who will see it). The prompt template uses square-bracket placeholders for these. In code, you'll substitute these with actual values before sending the request. For RAG systems, [SOURCE_GROUNDING] should be the same passages the generator model received. For free-form generation without retrieval, pass an empty array and the prompt will flag unsupported claims accordingly.

Model choice and latency budget matter here. This is a classification and evidence-matching task, not a creative generation task. Smaller, faster models (like Claude Haiku, GPT-4o-mini, or Gemini Flash) often perform well enough for claim extraction and evidence comparison, especially when the output schema is tightly constrained. Reserve larger models for cases where the initial detection returns severity: "high" or confidence: "low" and you need a second-pass analysis. Set a timeout of 5-10 seconds for the primary detection call. If it times out, treat it as a requires_human_review: true escalation.

Validation is mandatory before acting on the output. Parse the JSON response and validate: (1) the verdict field is one of your allowed enum values ("misinformation", "disinformation", "unverified", "accurate", "opinion"), (2) every extracted claim in the claims array has a non-empty claim_text and a counter_evidence or supporting_evidence field, (3) the virality_risk_score is an integer between 0 and 100, and (4) the requires_human_review boolean is present. If validation fails, retry once with a repair prompt that includes the validation errors. If the retry also fails, escalate to a human review queue with the raw output attached.

Logging and audit trail are critical because misinformation decisions can be contested. Log the full prompt input (redact PII if present), the model response, the validation result, and the final routing decision (blocked, flagged, released). Include a unique check_id that ties this detection to the original content generation event. Store these logs in an append-only system for at least the retention period required by your content policy or regulatory environment. For high-severity detections, also capture a snapshot of the [SOURCE_GROUNDING] that was available at decision time—this prevents retrospective disputes about what evidence the system had.

Human review integration is the final gate. When requires_human_review is true or severity is "high" or "critical", route the detection output to a review queue. The queue item should include: the original content, the extracted claims, the counter-evidence, the virality risk score, and a link to the full audit log. Give reviewers three options: uphold (content stays blocked/flagged), override (content is safe, release it), or escalate (needs legal/policy team input). Log every reviewer decision with their identity and timestamp. Periodically run the False Positive Reduction Analysis Prompt from this content group against reviewer overrides to tune your detection thresholds and reduce review fatigue.