Context Window Truncation and Hallucination Correlation Prompt

This prompt is designed for infrastructure and RAG engineers who have already confirmed a hallucination in a production output and suspect the root cause is context window truncation. The job-to-be-done is correlating specific truncated passages with specific hallucinated claims to produce a structured, evidence-backed report. The ideal user has access to a trace system that records the full retrieved context, the truncated context actually passed to the model, and the final generated output. Without this complete trace data, the prompt cannot perform a meaningful correlation and will produce unreliable results.

To use this prompt effectively, you must provide a single trace object containing three distinct data structures: the full list of retrieved documents before truncation, the subset of those documents that fit within the model's context window, and the final generated output containing the hallucinated claim. The prompt's instructions force the model to operate as a forensic analyst, not a general-purpose assistant. It requires the model to cite specific truncated passage IDs and map them to specific hallucinated claims using a structured output schema. This constraint prevents the model from offering vague speculation about why a hallucination occurred and instead demands traceable, auditable evidence. For high-stakes domains, the output should be reviewed by a human before any remediation is applied to the retrieval or chunking pipeline.

Do not use this prompt for real-time guardrails, as it is designed for post-hoc forensic analysis and is not optimized for latency. Do not use it when the hallucination cause is clearly a retrieval failure where relevant documents were never fetched, a source conflict where two retrieved documents contradict each other, or a prompt instruction leakage where the system prompt overrode the evidence. In those cases, use the sibling prompts for retrieval failure analysis, source conflict resolution, or instruction leakage detection. After running this prompt, the next step is typically to adjust your chunking strategy, context window budget, or reranking logic based on the specific truncated passages identified in the correlation report.

This prompt template is designed to be pasted directly into your trace analysis tool, evaluation pipeline, or LLM workbench. It instructs the model to act as a diagnostic auditor, comparing the full retrieved context against the truncated context that actually fit in the window, then mapping each hallucinated claim to the specific evidence that was lost during truncation. The output is a structured correlation report that infrastructure and RAG engineers can use to adjust chunking strategies, re-rankers, or context budgets.

textCopy
You are a trace diagnostic auditor specializing in RAG systems. Your task is to analyze a production trace where context window truncation occurred and determine whether the truncation caused specific hallucinations in the model's output.

## INPUTS
- Full Retrieved Context: [FULL_RETRIEVED_CONTEXT]
- Truncated Context (what actually fit in the window): [TRUNCATED_CONTEXT]
- Generated Output: [GENERATED_OUTPUT]
- Ground Truth or Expected Claims (optional): [GROUND_TRUTH_CLAIMS]

## INSTRUCTIONS
1. Extract every factual claim from the Generated Output. Number them sequentially.
2. For each claim, check whether it is supported by the Truncated Context. Mark as SUPPORTED, UNSUPPORTED, or PARTIALLY_SUPPORTED.
3. For every UNSUPPORTED or PARTIALLY_SUPPORTED claim, search the Full Retrieved Context for evidence that was lost during truncation.
4. If lost evidence is found, record the exact passage that was truncated and explain how it would have supported or corrected the claim.
5. If no evidence exists in either context, classify the claim as a MODEL-GENERATED HALLUCINATION (no grounding available).
6. Identify any claims that appear in the Truncated Context but were contradicted by truncated evidence in the Full Retrieved Context. Flag these as TRUNCATION-INDUCED CONTRADICTIONS.

## OUTPUT SCHEMA
Return a JSON object with this structure:
{
  "trace_summary": {
    "total_claims_extracted": <int>,
    "supported_by_truncated_context": <int>,
    "unsupported_claims_total": <int>,
    "truncation_correlated_hallucinations": <int>,
    "model_generated_hallucinations_no_grounding": <int>,
    "truncation_induced_contradictions": <int>
  },
  "claims_analysis": [
    {
      "claim_id": <int>,
      "claim_text": "<string>",
      "truncated_context_status": "SUPPORTED | UNSUPPORTED | PARTIALLY_SUPPORTED",
      "lost_evidence_found": <bool>,
      "lost_evidence_passage": "<string or null>",
      "lost_evidence_position": "<string or null, e.g., 'document_3_paragraph_5'>",
      "correlation_explanation": "<string>",
      "hallucination_category": "TRUNCATION-CORRELATED | MODEL-GENERATED | TRUNCATION-INDUCED-CONTRADICTION | NONE"
    }
  ],
  "truncation_impact_assessment": {
    "severity": "CRITICAL | HIGH | MEDIUM | LOW | NONE",
    "summary": "<string>",
    "recommended_actions": ["<string>"]
  }
}

## CONSTRAINTS
- Do not invent evidence. If a passage is not in the provided contexts, do not claim it exists.
- If the Ground Truth Claims are provided, use them to validate your analysis but do not treat them as retrieved context.
- Flag any claims where the Truncated Context appears to support the claim but the Full Retrieved Context reveals the support was misleading or incomplete.
- If the Generated Output contains no hallucinations, return an empty claims_analysis array and set severity to NONE.

To adapt this template for your environment, replace the square-bracket placeholders with data extracted from your trace store. The FULL_RETRIEVED_CONTEXT should include every document or chunk returned by your retrieval pipeline before truncation, while TRUNCATED_CONTEXT should contain only what survived your context window budget. If you use a re-ranker, the truncated context should reflect post-ranking order. For automated pipelines, wire this prompt into a batch evaluation job that runs nightly against sampled production traces, and route any CRITICAL severity findings to an incident channel. Always include a human review step before acting on HIGH or CRITICAL assessments, as the correlation analysis itself can contain errors.

This prompt is designed to be run as a post-hoc analysis step, not as a real-time guardrail. It should be triggered by an observability pipeline after a trace has been fully recorded and a hallucination has been flagged by an upstream evaluator or user report. The harness must assemble the full trace payload—including the final generated output, the list of retrieved chunks with their order and metadata, and the model's context window limit at the time of the request—before invoking the prompt. The prompt's primary value is in correlating specific truncated passages with specific unsupported claims, so the harness must preserve the exact truncation boundary: which chunks were included, which were dropped, and the token counts at the point of eviction.

Wire the prompt into a batch analysis job that listens for hallucination_detected events from your eval system. For each event, fetch the corresponding trace from your observability store (e.g., LangSmith, Arize, or a custom trace database). Construct the [TRACE_DATA] input as a structured JSON object containing three arrays: retrieved_chunks (with chunk_id, content, token_count, and included_in_context boolean), generated_claims (extracted by a prior claim extraction step), and truncation_events (marking where the context window limit forced eviction). The harness should validate that included_in_context is false for at least one chunk before running the prompt; if no truncation occurred, skip the analysis and log a no_truncation status. Use a model with a large context window (e.g., Claude 3.5 Sonnet or GPT-4o) to ensure the prompt itself can ingest the full trace without secondary truncation. Set temperature=0 and enforce structured output using the [OUTPUT_SCHEMA] defined in the prompt template—a JSON array of correlation objects, each linking a truncated chunk ID to a generated claim ID with a correlation_strength score and rationale.

After the prompt returns, validate the output against the input trace: every truncated_chunk_id must reference a chunk where included_in_context is false, and every generated_claim_id must exist in the input claims array. Discard correlations with correlation_strength below a configurable threshold (start at 0.7) and log them as weak_correlation_discarded for later review. For high-severity hallucinations—those flagged as critical or major by your severity classifier—route the correlation report to a human review queue before accepting it as a root-cause finding. Store the validated correlation report back into your trace store, linked to the original trace ID, so that future incident reviews can surface the truncation-to-hallucination chain without re-running the analysis. Avoid running this prompt on every request; it is expensive in both tokens and latency, and should be reserved for diagnosed hallucinations where context window pressure is a plausible contributing factor.