Paraphrase Drift Hallucination Check Prompt Template

This prompt is designed for summarization and synthesis quality teams who need to catch a specific, insidious class of hallucination: meaning changes introduced during paraphrasing. Unlike outright fabrication, paraphrase drift occurs when a model re-expresses a source concept but subtly alters its intensity, scope, or core claim. The primary job-to-be-done is to compare a generated paraphrase or summary sentence against its source evidence span and flag any semantic drift that would mislead a downstream reader. The ideal user is an AI output validation engineer or a quality assurance lead building an automated guardrail that sits between content generation and user delivery.

Use this prompt when your application generates summaries, rewrites, or synthesized answers from source material and you need to verify that the output remains faithful to the original evidence. It is particularly valuable in high-stakes domains—legal summarization, clinical note synthesis, financial report generation—where a slight shift in wording can alter meaning with material consequences. The prompt expects two inputs: the original source span and the generated paraphrase. It returns a structured drift assessment with a severity scale, making it suitable for integration into automated evaluation pipelines, CI/CD checks for prompt changes, or human review queues where flagged outputs are escalated.

Do not use this prompt for detecting complete fabrications or checking whether a statement is supported by any evidence at all. Those jobs belong to sibling playbooks like Unsupported Statement Detection or Fabricated Detail Detection. This prompt assumes the generated text is grounded in the provided source—it only checks whether the grounding has warped during rewording. Also avoid using it for long-form outputs where source-to-sentence alignment hasn't been established first; run a sentence-level grounding verification step before applying drift checks. If your risk tolerance is zero for meaning alteration, always pair this automated check with human review for outputs flagged at moderate severity or above.

This prompt template is designed to compare a generated paraphrase or summary against its source span and flag instances where the meaning has shifted. It is built for integration into automated output validation pipelines where summarization and synthesis quality must be verified before responses reach users. The template uses square-bracket placeholders for all variable inputs, making it straightforward to adapt for different source materials, risk tolerances, and output schemas.

textCopy
You are a precise semantic auditor. Your task is to compare a generated paraphrase against its original source text and detect any meaning changes, exaggerations, omissions, or fabrications introduced during paraphrasing.

## INPUT
**Original Source Text:**
[SOURCE_TEXT]

**Generated Paraphrase:**
[PARAPHRASE_TEXT]

## INSTRUCTIONS
1. Read the original source text carefully to understand its precise meaning, scope, and limitations.
2. Read the generated paraphrase and identify every claim, statement, or implication it contains.
3. For each claim in the paraphrase, determine whether it is:
   - **FULLY SUPPORTED**: The claim matches the source's meaning exactly, even if wording differs.
   - **DRIFTED**: The claim alters, exaggerates, softens, or shifts the source's meaning in any way.
   - **FABRICATED**: The claim introduces information not present in or inferable from the source.
   - **OMITTED**: Material information from the source is absent from the paraphrase.
4. For each DRIFTED or FABRICATED claim, assign a severity level:
   - **MINOR**: Cosmetic wording change that does not materially alter understanding.
   - **MODERATE**: Meaning shift that could mislead or change interpretation.
   - **CRITICAL**: Meaning shift that reverses, contradicts, or dangerously misrepresents the source.
5. For each OMITTED item, assess whether the omission changes the overall message.

## CONSTRAINTS
- Do not flag stylistic differences or synonym substitutions that preserve meaning.
- Do flag any shift in certainty, scope, attribution, causality, or quantification.
- If the paraphrase adds qualifiers (e.g., "might," "possibly") where the source was definite, flag it as DRIFTED.
- If the paraphrase removes qualifiers where the source was uncertain, flag it as DRIFTED.
- If the source contains [DOMAIN_TERMS], pay special attention to domain-specific precision.

## OUTPUT_SCHEMA
Return a JSON object with the following structure:
{
  "overall_drift_score": <float between 0.0 and 1.0, where 0.0 = no drift and 1.0 = complete meaning reversal>,
  "drift_detected": <boolean>,
  "findings": [
    {
      "paraphrase_span": "<exact text from paraphrase>",
      "source_span": "<corresponding text from source or null if fabricated>",
      "finding_type": "DRIFTED | FABRICATED | OMITTED",
      "severity": "MINOR | MODERATE | CRITICAL",
      "explanation": "<precise description of the semantic difference>"
    }
  ],
  "summary": "<one-sentence overall assessment>"
}

## EXAMPLES
[EXAMPLES]

## RISK_LEVEL
[RISK_LEVEL]

To adapt this template, replace each square-bracket placeholder with your specific inputs. [SOURCE_TEXT] and [PARAPHRASE_TEXT] are required for every invocation. [DOMAIN_TERMS] should be populated with a comma-separated list of domain-specific terminology that requires precise handling—for example, medical terms, legal phrases, or technical specifications. [EXAMPLES] should contain one to three few-shot demonstrations showing correct drift detection across different severity levels, which significantly improves model calibration. [RISK_LEVEL] accepts values like LOW, MEDIUM, HIGH, or CRITICAL and should be used to adjust the strictness of detection—higher risk levels should trigger more conservative flagging behavior. When wiring this into a production pipeline, always validate the output JSON against the schema before accepting results, and consider running multiple passes with different temperature settings to catch borderline drift cases.

The Paraphrase Drift Hallucination Check prompt is designed to operate as a post-generation guardrail within a summarization or synthesis pipeline. It should be invoked after the primary model generates a summary or paraphrase, receiving both the generated output and the original source span as inputs. The prompt is not a real-time streaming interrupter; it works best on complete text segments where semantic drift can be assessed holistically. In a typical RAG or document-processing architecture, this check sits between the generation step and the final response delivery, ensuring that meaning-preserving rewrites are not incorrectly flagged as hallucinations while genuine distortions are caught before they reach the user.

To integrate this prompt into an application, build a dedicated validation service that accepts a source_span and a generated_paraphrase as inputs. The service should call the LLM with the prompt template, requesting a structured JSON output that includes a drift_detected boolean, a drift_severity score (e.g., none, minor, material, severe), a drift_type classification (e.g., exaggeration, omission, meaning_change, fabrication), and a drift_explanation string. Validation of the validator is critical here: implement a second-pass check that verifies the JSON schema is well-formed and that the drift_severity label is consistent with the explanation provided. If the model returns a severe drift but the explanation describes only a minor synonym substitution, flag this inconsistency for human review rather than blocking the output automatically. For high-stakes domains like medical or legal summarization, always route material and severe drift detections to a human review queue before taking action.

Model choice matters for this check. Use a capable instruction-following model (such as GPT-4, Claude 3.5 Sonnet, or Gemini 1.5 Pro) because the task requires fine-grained semantic comparison. Smaller or faster models often over-flag stylistic paraphrasing as drift or, conversely, miss subtle meaning alterations. Set temperature=0 to maximize consistency across repeated checks. Implement a retry strategy for malformed JSON outputs: if the response fails to parse against the expected schema, retry once with the same prompt and a stronger schema reminder. If the second attempt also fails, log the raw output and escalate to a human operator rather than silently passing the content. For observability, log every drift check result—including the source span, generated text, drift severity, and the validator model's response—into your tracing system. This data becomes invaluable for tuning retrieval precision, adjusting generation prompts, and identifying systematic drift patterns across document types or model versions.