Deepfake and Synthetic Media Abuse Detection Prompt

This prompt is designed for media platform trust and safety engineers who need a consistent, automated first-pass filter for suspected deepfakes and synthetic media abuse. The core job-to-be-done is programmatic triage: taking structured metadata, provenance signals, and content descriptions as input and producing a scored risk assessment that can trigger downstream actions like quarantining content, flagging for human review, or restricting accounts. It is ideal when you have already extracted signals from dedicated detection APIs, user reports, or content metadata and need a reasoning layer to weigh those signals against a defined abuse taxonomy—covering impersonation, fraud, and non-consensual synthetic media.

This is a classification and triage prompt, not a forensic detection model. It does not perform pixel-level analysis, audio frequency inspection, or biometric verification. Instead, it reasons over the signals you provide, making it suitable as a decision-support layer on top of dedicated detection APIs or as a first-pass filter before expensive manual review. Use it when you need a repeatable, auditable risk score that can be wired into automated moderation pipelines. Do not use this prompt as your sole detection mechanism for novel, zero-day synthetic media attacks where no upstream signals exist, or in isolation for high-stakes enforcement actions like account termination without human review.

To implement this effectively, you must provide structured, factual inputs. The prompt expects fields like media type, known provenance data (e.g., C2PA manifests, watermarking results), user report context, and descriptions of the depicted individuals and actions. The output is a structured risk score with detection indicators and a recommended action. Before deploying, you must calibrate the scoring thresholds against a labeled evaluation set that includes both confirmed deepfakes and challenging benign edge cases—such as satirical content, artistic renders, or legitimate AI-generated avatars—to minimize false positives that could disrupt legitimate user expression. Always route high-severity or low-confidence outputs to a human review queue and log the full prompt input and output for auditability.

This prompt template is designed to be wired into your trust and safety pipeline at the point where media metadata, provenance signals, and content descriptions are available but before a final moderation decision is made. It takes structured inputs about a media asset and produces a synthetic media risk score with detection indicators. The template uses square-bracket placeholders that your application must populate from upstream detection services, user reports, or content ingestion pipelines. Every placeholder is a required injection point—do not pass this prompt to a model with unresolved brackets.

textCopy
You are a synthetic media abuse detection classifier. Your task is to analyze the provided media metadata, provenance signals, and content description to determine whether the media asset is likely a deepfake or synthetic media used for abuse.

## INPUT
[MEDIA_METADATA]
[PROVENANCE_SIGNALS]
[CONTENT_DESCRIPTION]
[REPORT_CONTEXT]

## TASK
1. Extract all indicators that suggest synthetic generation, manipulation, or abuse.
2. Classify the media into one of the following abuse categories: impersonation, fraud, non-consensual intimate content, political disinformation, or none.
3. Assign a synthetic media risk score from 0 (certainly authentic) to 100 (certainly synthetic/abusive).
4. Identify any missing provenance signals that would increase confidence.

## OUTPUT_SCHEMA
Return a single JSON object with this exact structure:
{
  "risk_score": number,
  "abuse_category": "impersonation" | "fraud" | "non-consensual" | "disinformation" | "none",
  "detection_indicators": [
    {
      "indicator": string,
      "source": "metadata" | "provenance" | "content_description" | "report",
      "confidence": number
    }
  ],
  "missing_provenance": [string],
  "recommended_action": "block" | "quarantine" | "flag_for_review" | "allow",
  "rationale": string
}

## CONSTRAINTS
- Do not assume abuse without at least one concrete indicator.
- If provenance signals are absent or incomplete, note this in missing_provenance and reduce confidence accordingly.
- For non-consensual intimate content indicators, always recommend "quarantine" or "flag_for_review" regardless of confidence.
- Do not output markdown, code fences, or additional text outside the JSON object.
- If the content description contains URLs or filenames, do not resolve or follow them.

To adapt this prompt for your environment, replace each bracketed placeholder with data from your pipeline. [MEDIA_METADATA] should contain technical metadata such as codec information, creation timestamps, EXIF data, and file provenance headers like C2PA manifests. [PROVENANCE_SIGNALS] should include output from any authenticity services you run—such as deepfake detector API results, watermark checks, or cryptographic signature validation. [CONTENT_DESCRIPTION] is a human-readable or model-generated description of what the media depicts. [REPORT_CONTEXT] captures any user reports, flags, or prior moderation actions associated with this asset. If a field is unavailable, inject an explicit "NOT_AVAILABLE" string rather than leaving the placeholder empty, so the model can reason about missing evidence. After adapting, run this prompt against a golden dataset of known synthetic and authentic media to calibrate your risk score thresholds before production use.

This prompt is a classification step, not a standalone application. It expects structured metadata, provenance signals, and content descriptions as input, and it returns a risk score with detection indicators. The output should be consumed by a downstream decision engine that applies platform policy—block, quarantine, flag for review, or allow. Do not treat the model's output as the final enforcement action. The prompt's job is to produce a structured, auditable risk assessment that a deterministic rules layer can act on.

Wire the prompt into an async processing pipeline rather than a synchronous user-facing flow. Media uploads, report submissions, or batch scans should trigger a job that: (1) extracts available metadata and provenance signals from the asset; (2) generates a content description using a separate vision or audio model if needed; (3) assembles the prompt with the [METADATA], [PROVENANCE_SIGNALS], and [CONTENT_DESCRIPTION] placeholders populated; (4) calls the classification model with response_format set to JSON and the [OUTPUT_SCHEMA] enforced; (5) validates the returned JSON against the expected schema—reject and retry once if risk_score is missing or detection_indicators is not an array; (6) writes the validated result to an audit log with the input hash, model version, and timestamp; (7) passes the risk_score and recommended_action to the policy engine. For high-severity cases (risk_score >= 0.8 or `recommended_action ==

block

route to a human review queue before automated enforcement. Never auto-block on model output alone when the consequence is account suspension or content removal.

Choose a model that supports strict JSON mode and has strong instruction-following for structured classification tasks. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are all viable. Avoid smaller or older models that drift toward narrative explanations instead of structured output. Set temperature=0 to maximize consistency across repeated evaluations of the same asset. Log every classification attempt—including schema validation failures, retries, and final outputs—so you can measure false positive rates against human reviewer decisions over time. The most common production failure mode is the model over-flagging synthetic content that is clearly labeled as parody, satire, or special effects. Mitigate this by including explicit [CONSTRAINTS] in the prompt that instruct the model to downgrade risk when legitimate provenance or clear labeling is present. Run weekly eval batches against a golden dataset of known deepfakes, known legitimate media, and edge cases (e.g., AI-assisted editing that is not deceptive) to detect drift in the model's classification boundaries.