Misinformation Generation Intent Detection Prompt

This prompt is built for safety engineers and platform operators who need to detect user requests that seek to generate deceptive or harmful misinformation. It is designed for high-stakes contexts such as elections, public health, and civic information where the cost of a miss is severe. The prompt classifies the intent behind a user request, not the truth of a claim. It distinguishes between a user asking for help drafting a misleading narrative and a user asking for factual information about a contested topic. Use this prompt as an upstream safety gate before a model generates content. It belongs in a pre-generation classification layer, not in a post-hoc fact-checking pipeline.

The prompt includes topic-specific risk weighting so that a request about vaccine safety in a health context receives different treatment than a request about a fictional story. It also enforces political neutrality by focusing on deceptive intent rather than viewpoint disagreement. This means a request to 'write an article arguing that climate change is a hoax' would be flagged for deceptive intent, while a request to 'summarize the scientific consensus on climate change' would not, even if the user holds a minority viewpoint. The classification targets the user's demonstrated intent to create misleading content, not the political valence of the topic. Wire this prompt into your request pipeline before any generation model, and configure your routing logic to block, flag, or escalate based on the classification output and your organization's risk thresholds.

Do not use this prompt as a standalone fact-checker or as a replacement for human content moderation. It is not designed to evaluate the truth of claims, verify citations, or assess the accuracy of generated content. For those tasks, use a dedicated fact-checking or claims verification prompt from the Grounding and Evidence Ranking or Fact Checking and Claims Verification playbooks. Also avoid using this prompt in low-stakes creative writing contexts where users routinely request fictional scenarios that mimic real-world misinformation patterns—the prompt will generate too many false positives. Deploy this prompt when the cost of generating deceptive content outweighs the cost of occasional over-refusal, and always pair it with a human review path for borderline classifications.

This template is designed to be placed in your system prompt or as a dedicated classification step before your main model responds. It detects requests that seek to generate misleading, fabricated, or deceptive content, with special attention to electoral, health, and civic information integrity. The prompt uses square-bracket placeholders that you must replace with your domain taxonomy, risk thresholds, and output schema before deployment.

textCopy
You are a misinformation intent classifier. Your job is to analyze user requests and determine whether they seek to generate deceptive, misleading, or fabricated content. You do not evaluate the truth of claims in the request itself. You evaluate the user's apparent intent to create content designed to deceive.

## RISK WEIGHTING
Apply higher scrutiny to requests involving these domains:
[DOMAIN_RISK_WEIGHTS]

## CLASSIFICATION TAXONOMY
Classify each request into exactly one of these categories:
[INTENT_CATEGORIES]

## INPUT
User request: [USER_REQUEST]
Conversation context (previous turns): [CONVERSATION_CONTEXT]
User role or permissions: [USER_CONTEXT]

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "classification": "[one of the intent categories]",
  "confidence": [float between 0.0 and 1.0],
  "risk_score": [integer between 1 and 10],
  "triggered_domains": ["list of domains from DOMAIN_RISK_WEIGHTS that are relevant"],
  "rationale": "Brief explanation of the classification decision",
  "uncertainty_flags": ["list of any factors that made this classification uncertain"],
  "recommended_action": "[ALLOW | FLAG | BLOCK | ESCALATE]"
}

## CONSTRAINTS
[CONSTRAINTS]

## EXAMPLES
[FEW_SHOT_EXAMPLES]

## POLITICAL NEUTRALITY
You must apply the same classification standards regardless of the political orientation, nationality, or affiliation of the content. Classification must be based on deceptive intent, not viewpoint disagreement. If a request could be interpreted as legitimate political speech or satire, flag it with uncertainty and recommend human review.

## FACTUAL CLAIM VERIFICATION BOUNDARY
You are NOT a fact-checker. You do not verify whether specific claims are true or false. You classify the intent to deceive based on the framing, context, and explicit signals in the request. If a request asks for content that would be deceptive regardless of factual accuracy, classify accordingly. If you cannot distinguish between a legitimate request for information and a request to generate misinformation, flag the uncertainty and recommend human review.

To adapt this template, replace each square-bracket placeholder with your operational values. [DOMAIN_RISK_WEIGHTS] should list domains like electoral processes, public health, civic infrastructure, and financial markets, each with a numeric weight that influences the risk score. [INTENT_CATEGORIES] should define your classification labels such as deliberate_fabrication, deceptive_editing, false_context, impersonation_content, legitimate_request, and ambiguous. [CONSTRAINTS] should include any additional rules such as handling of satire, educational contexts, or research requests. [FEW_SHOT_EXAMPLES] must include at least three positive examples of misinformation intent and three negative examples of legitimate requests that could be mistaken for harmful intent. Before deploying, validate that your output schema matches your downstream enforcement logic and that your risk thresholds are calibrated against a labeled evaluation set.

This prompt is designed to sit inside a pre-generation safety gateway, not as a post-hoc moderation check. The system should call this prompt before the main model generates a response to a user request. The input is the raw user query plus any retrieved context that would be provided to the downstream model. The output is a structured classification that your application layer must parse and act on: block, flag_for_review, or allow. Because misinformation detection is inherently contextual, the prompt requires topic-specific risk weighting via the [TOPIC_CONTEXT] placeholder—this should be populated dynamically based on the conversation domain (e.g., electoral, health, civic).

Integration pattern: Wrap this prompt in a lightweight API service or middleware function that intercepts user requests before they reach the main generation endpoint. The service should: (1) assemble the prompt with the user's raw input and the current topic context; (2) call a fast, cost-efficient model (GPT-4o-mini, Claude Haiku, or a fine-tuned classifier) with temperature=0 and response_format set to JSON mode; (3) parse the classification field from the response; (4) route based on the result. For block decisions, return a canned safe-refusal response immediately. For flag_for_review decisions, enqueue the request to a human review queue with the full prompt payload, classification metadata, and a TTL for review (e.g., 15 minutes for real-time chat). For allow decisions, pass the original request through to the main generation pipeline. Validation: Before acting on the classification, validate that the response contains all required fields (classification, confidence_score, risk_category, rationale), that classification is one of the allowed enum values, and that confidence_score is a float between 0 and 1. If validation fails, default to flag_for_review and log the malformed response for debugging.

Retry and fallback logic: If the classification call fails due to a network error, timeout, or model unavailability, implement up to 2 retries with exponential backoff (e.g., 200ms, 800ms). If all retries fail, the system must fail closed: route the request to flag_for_review and log the outage. Do not allow requests to bypass the safety gateway on error. For latency-sensitive applications, set a strict timeout of 800ms on the classification call; if it exceeds this budget, treat it as a timeout failure and follow the same fail-closed path. Logging and observability: Log every classification decision with the following fields: request_id, timestamp, user_input_hash (not raw input, for privacy), topic_context, classification, confidence_score, risk_category, model_used, latency_ms, retry_count, and routing_decision. This log becomes your audit trail for safety decisions and your dataset for calibrating confidence thresholds over time.

Human review routing: When a request is flagged for review, the review interface should display: the original user input, the topic context, the model's classification and rationale, and the confidence score. Reviewers should have three actions: confirm_block, override_allow, or escalate. Every reviewer action must be logged with the reviewer ID and timestamp. Use these human decisions to build a labeled dataset for evaluating and fine-tuning your classification threshold. Model choice: Start with a fast classifier model. If you observe high false-positive rates on legitimate civic discussion or health questions, consider moving to a larger model for the classification step or implementing a two-stage pipeline where low-confidence decisions from the fast model are re-evaluated by a more capable model before routing to human review.

What to avoid: Do not use this prompt as the sole defense for generated content—it only classifies user intent, not model outputs. You still need output-side factuality checks and citation grounding for the generation step. Do not hardcode the [TOPIC_CONTEXT] to a single domain; the risk weighting must shift based on what the user is actually discussing. Do not skip the validation step; malformed JSON from the classifier can crash your routing logic. Finally, do not treat the confidence score as a probability in the strict statistical sense—it is a model-generated signal that requires calibration against your own labeled data before you can set reliable thresholds for automated blocking.