Summarize-Then-Continue Prompt for Long Documents

This playbook is for AI engineers and application developers who need to process documents larger than a single context window. The two-stage strategy first summarizes processed sections into a dense state payload, then continues with the remaining content while preserving key entities, relationships, and facts. Use this when a document is too large for one pass and you cannot simply truncate or skip sections. This prompt belongs in document intelligence pipelines, long-form report generation, contract review systems, and any workflow where fact retention across the context boundary is critical.

The ideal user is building a production system where document completeness matters. For example, a legal tech platform processing 200-page contracts needs every clause examined, not just the first 50 pages. A financial analyst tool reviewing annual reports must track entity relationships and numeric values across sections. An academic research assistant synthesizing multiple papers needs to preserve citation chains and methodology details. In each case, the Summarize-Then-Continue prompt provides a structured handoff between processing windows, carrying forward a compressed but faithful representation of what was already read. The prompt template expects a [DOCUMENT_SECTION] input, a [PREVIOUS_SUMMARY] state payload, and an [EXTRACTION_SCHEMA] defining what entities, facts, and relationships to preserve. The output is a continuation analysis plus an updated summary ready for the next window.

Do not use this prompt for real-time chat or short-form content where a sliding window summary is more appropriate. If your documents fit comfortably within a single context window, a standard extraction or analysis prompt is simpler and less error-prone. Avoid this strategy when the task requires cross-referencing arbitrary sections that may be far apart—the summary state is lossy by design, and critical connections can be missed. Also avoid it when latency is the primary constraint; the two-stage handoff adds processing overhead. For high-risk domains like healthcare or legal, always include a human review step after the full document is processed, and run eval checks comparing the final output against spot-checked original sections to measure fact retention accuracy. Before implementing, define your retry budget: if the model produces an incomplete summary that breaks the continuation, how many times will you re-prompt before escalating to a human operator or splitting the document differently?

Next, review the prompt template section to copy the base instruction and adapt the placeholders to your document type and extraction requirements. Then move to the implementation harness to wire the two-stage loop into your application with proper validation, logging, and failure recovery.

This template implements a summarize-then-continue pattern for documents that exceed the model's context window. Stage 1 processes a chunk of content and produces a structured summary capturing key entities, relationships, claims, and the boundary point where processing stopped. Stage 2 receives that summary as preserved state alongside the next chunk, allowing the model to continue analysis without re-reading the original text. The template is designed for sequential processing pipelines where each chunk's output feeds the next chunk's input.

codeCopy
STAGE 1: SUMMARIZE PROCESSED CONTENT

You are processing a long document in chunks. Your job is to read the current chunk, extract structured information, and produce a summary that will be passed to the next processing stage.

## INPUT

[CHUNK_TEXT]

## PREVIOUS STATE (if any)

[PREVIOUS_SUMMARY or "This is the first chunk."]

## INSTRUCTIONS

1. Read the current chunk and identify:
   - Named entities (people, organizations, locations, products)
   - Key claims or findings with their supporting evidence
   - Relationships between entities
   - Dates, numbers, and metrics
   - Decisions, action items, or conclusions
   - The exact boundary text where this chunk ends

2. Produce a structured summary in the following format:

## OUTPUT SCHEMA

{
  "chunk_index": [CHUNK_INDEX],
  "entities": [
    {
      "name": "string",
      "type": "person|organization|location|product|other",
      "mentions": ["relevant context from chunk"],
      "relationships": ["relationship descriptions"]
    }
  ],
  "claims": [
    {
      "claim": "string",
      "evidence": "supporting text from chunk",
      "confidence": "high|medium|low"
    }
  ],
  "metrics": [
    {
      "name": "string",
      "value": "string or number",
      "context": "surrounding context"
    }
  ],
  "decisions_and_actions": ["list of decisions or action items"],
  "boundary_marker": "exact last sentence or phrase processed",
  "unresolved_questions": ["questions raised but not yet answered"],
  "narrative_summary": "2-3 sentence prose summary of this chunk"
}

## CONSTRAINTS

- Preserve exact numbers, dates, and proper nouns without paraphrasing.
- Mark claims as low confidence if evidence is ambiguous or incomplete.
- Include the boundary marker verbatim to enable exact continuation.
- Do not invent entities or claims not present in the chunk.
- If this is not the first chunk, integrate new entities with those from the previous summary.

---

STAGE 2: CONTINUE WITH NEXT CHUNK

You are continuing to process a long document. You have a summary of all previously processed content. Use it to maintain continuity while processing the next chunk.

## PREVIOUS STATE SUMMARY

[STAGE_1_OUTPUT]

## NEXT CHUNK

[NEXT_CHUNK_TEXT]

## INSTRUCTIONS

1. Review the previous state summary to understand:
   - All entities, claims, and relationships discovered so far
   - Unresolved questions that this chunk might answer
   - The exact boundary where previous processing stopped

2. Process the next chunk, updating the state:
   - Add new entities and merge with existing ones when the same entity appears
   - Add new claims and note if they resolve prior unresolved questions
   - Update relationships as new connections emerge
   - Track metrics across chunks (cumulative totals, trends)
   - Record new decisions and actions

3. Produce an updated summary using the same schema as Stage 1, but with cumulative state:

## OUTPUT SCHEMA

{
  "chunk_index": [CHUNK_INDEX],
  "cumulative_entities": [/* merged entity list */],
  "cumulative_claims": [/* all claims across chunks */],
  "cumulative_metrics": [/* all metrics with cross-chunk context */],
  "cumulative_decisions_and_actions": [/* all decisions and actions */],
  "boundary_marker": "exact last sentence processed in this chunk",
  "resolved_questions": ["previously unresolved questions now answered"],
  "new_unresolved_questions": ["new questions raised in this chunk"],
  "cross_chunk_narrative": "2-3 sentence summary connecting this chunk to prior content",
  "processing_complete": false
}

## CONSTRAINTS

- Do not re-process content from previous chunks; rely on the provided summary.
- When the same entity appears across chunks, merge information under one entry.
- Flag contradictions between chunks if claims conflict.
- Set processing_complete to true only when the final chunk is processed.
- If the chunk appears to repeat prior content, note the overlap rather than duplicating entries.

Adapt this template by adjusting the output schema to match your domain's entity types, claim structures, and metric categories. For legal documents, add fields for clauses, obligations, and parties. For clinical documents, add fields for medications, conditions, and procedures. For financial documents, add fields for transactions, accounts, and compliance flags. The boundary marker is critical for resumability—always capture the exact text where processing stopped. If your pipeline processes more than two chunks, repeat Stage 2 for each subsequent chunk, passing the cumulative summary forward each time. Add a final Stage 3 if you need to produce a document-level synthesis from all cumulative summaries.

Before deploying this prompt into a production pipeline, build validation checks that verify the boundary marker matches the actual end of the chunk, entities are not hallucinated by cross-referencing against source text, and cumulative state does not grow unbounded across many chunks. For high-stakes domains like legal or clinical review, require human spot-checking of entity merges and claim confidence scores. Test with documents that contain deliberate cross-chunk references—entities introduced in chunk 1 and referenced again in chunk 3—to verify that the summary preserves enough state for correct resolution. The most common failure mode is entity duplication when the same person or organization appears with slightly different names across chunks; add normalization instructions to your schema if this risk is high.

The Summarize-Then-Continue pattern requires a two-stage harness that manages state between the summarization pass and the continuation pass. The application layer—not the model—must track which document sections have been processed, store the intermediate summary, and inject it into the continuation prompt. This is not a single prompt call; it is a stateful pipeline where the output of stage one becomes a critical input to stage two. The harness should treat the summary as a structured artifact with a schema (entities, relationships, key claims, and a section pointer) rather than a free-text blob, because the continuation prompt needs to reference specific facts without ambiguity.

Wire the pipeline as follows: (1) Split the source document into chunks that fit within your model's context window, reserving at least 30% of the window for the summary and continuation instructions. (2) Process chunks sequentially, maintaining a running summary object. After each chunk, call the summarization prompt with the chunk text and the existing summary, producing an updated summary. (3) When the final chunk is processed, call the continuation prompt with the complete summary and the original task instruction. Validate the summary at each step: check that entity counts are non-decreasing, that relationship edges reference entities present in the summary, and that the last_processed_position field advances monotonically. If validation fails, retry the summarization step with a repair instruction before proceeding. Log every summary update with a timestamp and chunk index for auditability.

For model choice, prefer models with large context windows and strong instruction-following for the summarization stage (Claude 3.5 Sonnet or GPT-4o), and use the same or a faster model for the continuation stage depending on output complexity. Set temperature=0 or very low for summarization to maximize factual consistency across chunks. Implement a retry budget: three attempts per chunk for summarization failures, then escalate to a human operator with the partial summary and the failed chunk. For the continuation stage, validate that the final output references only entities and claims present in the summary—any hallucinated addition indicates a summary fidelity problem upstream. Store the full summary artifact alongside the final output for downstream evaluation and debugging. Avoid the common failure mode of letting the summary grow unbounded; if the summary itself approaches the context limit, apply the same compression pattern recursively.