Redundant Section Collapse Prompt for Long-Form Output

This prompt is designed for a specific job: collapsing redundant sections within a single, long-form model output to improve coherence and reduce reader fatigue. The ideal user is a developer or engineer maintaining a documentation generation, report synthesis, or content assembly pipeline. You have a model that produces a complete document, but it often repeats the same information across multiple sections or subsections. Instead of manually editing the output or discarding it, you use this prompt as a post-generation repair step to merge overlapping content into a single, comprehensive section. The required context is the full, unedited long-form text, along with any structural constraints like a desired heading hierarchy or a list of sections that must be preserved. You should not use this prompt when the repetition is intentional for emphasis, when the output is a list of distinct items where semantic overlap is expected, or when the document's argumentative structure depends on revisiting a topic from different angles. In those cases, a deduplication or summarization prompt would be more appropriate.

The prompt works by instructing the model to act as a structural editor. It must first analyze the document to identify sections with high semantic overlap, then merge their content into a single, logically placed section, and finally verify that no critical information was lost in the collapse. The key operational constraint is maintaining document coherence. A naive collapse can break the narrative flow, remove necessary context, or create a disjointed reading experience. Therefore, the prompt template includes explicit instructions for preserving logical transitions and re-anchoring any cross-references. For example, if both a 'System Overview' section and a 'Key Components' subsection describe the same database architecture, the prompt should guide the model to merge them under the most appropriate heading, integrate all unique details, and update any downstream references like 'as mentioned in the System Overview.' The output should be the complete, restructured document, not just a list of changes.

Before integrating this prompt into a production pipeline, you must define clear evaluation criteria. A successful collapse is not just about removing text; it's about preserving factual completeness and improving readability. Your test suite should include documents with known redundant sections and a human-validated 'golden' collapsed version. Automated evals can check for a reduction in total token count and an increase in a semantic similarity score against the original document to ensure coverage. However, a human review step is critical for high-stakes documentation to confirm that the new structure is logically sound and that no nuanced information was lost. Start by running this prompt on a batch of historical outputs and comparing the results against your golden dataset to calibrate your acceptance thresholds before enabling it in an automated pipeline.

This prompt template is designed to be dropped into a post-generation repair pipeline. It takes a long-form document output from a model—such as a generated report, documentation page, or research synthesis—and identifies sections or subsections that repeat the same information. The prompt instructs the model to collapse those redundant sections into a single merged section, preserving all unique coverage while eliminating repetition. Use this when your generation pipeline produces verbose outputs with structural redundancy, not when you need simple sentence-level deduplication.

textCopy
You are a document structure editor. Your task is to analyze a long-form document and identify sections or subsections that repeat the same information. When you find redundant sections, collapse them into a single merged section that preserves all unique coverage from the originals.

## INPUT DOCUMENT
[INPUT_DOCUMENT]

## CONSTRAINTS
- Preserve the original document's logical flow and section hierarchy.
- Do not remove any unique information, facts, claims, or examples.
- When merging sections, place the merged section at the position of the first occurrence unless [MERGE_POSITION_RULE] specifies otherwise.
- Update all cross-references and internal links to point to the merged section.
- If two sections partially overlap, merge the overlapping content and keep distinct content in separate sections.
- Do not alter the document's introduction, conclusion, or navigation structure unless they contain redundant content.
- Maintain the original heading levels and numbering scheme.

## OUTPUT FORMAT
Return a JSON object with this schema:
{
  "collapsed_document": "[FULL_MERGED_DOCUMENT_TEXT]",
  "changes": [
    {
      "action": "merged" | "removed" | "kept",
      "source_sections": ["[SECTION_ID_OR_HEADING]"],
      "target_section": "[SECTION_ID_OR_HEADING]",
      "rationale": "[WHY_THESE_SECTIONS_WERE_REDUNDANT]",
      "unique_content_preserved": ["[LIST_OF_UNIQUE_POINTS_RETAINED]"]
    }
  ],
  "structural_integrity_checks": {
    "heading_hierarchy_intact": true | false,
    "cross_references_updated": true | false,
    "no_lost_coverage": true | false,
    "issues_found": ["[ANY_STRUCTURAL_PROBLEMS]"]
  }
}

## REDUNDANCY THRESHOLD
Consider sections redundant when they share [SIMILARITY_THRESHOLD]% or more of their substantive content. Substantive content means facts, claims, instructions, or conclusions—not transitional phrases or structural boilerplate.

## EXAMPLES
[FEW_SHOT_EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

Adapt this template by adjusting the [SIMILARITY_THRESHOLD] based on your tolerance for near-duplicate content. For technical documentation, set it high (80-90%) to avoid merging sections that share terminology but cover different aspects. For narrative reports, a lower threshold (60-70%) may be appropriate. The [MERGE_POSITION_RULE] placeholder accepts values like first_occurrence, last_occurrence, or most_complete_section. Provide [FEW_SHOT_EXAMPLES] showing your expected merge behavior for your document type. Set [RISK_LEVEL] to high when the document is regulatory, clinical, or legal—this should trigger downstream human review of all merges before publication.

The Redundant Section Collapse Prompt is designed to operate as a post-generation repair step within a long-form document pipeline. After a model produces a report, article, or documentation draft, this prompt receives the full output and returns a collapsed version where sections that repeat the same information are merged into a single coherent section. The harness must handle the full lifecycle: receiving the raw output, calling the collapse prompt, validating structural integrity, and either accepting the result or retrying with adjusted constraints.

Pipeline integration follows a standard repair-loop pattern. First, the raw model output is passed to the collapse prompt with the required [FULL_DOCUMENT] placeholder populated. The prompt returns a restructured document. Before accepting it, the harness must run structural integrity checks: verify that all original section anchors (if provided via [SECTION_ANCHORS]) still appear in the output, confirm that no new sections were hallucinated, and ensure the total word count hasn't dropped below a configured threshold (e.g., 70% of original length to guard against over-collapse). If validation fails, the harness retries with a modified [CONSTRAINTS] block that explicitly lists the failure—for example, "Retry: the previous collapse removed Section 3 entirely. Preserve all section anchors." Set a maximum of 2 retries before flagging for human review.

Model choice and tool use matter here. This prompt works best with models that have strong instruction-following and long-context capabilities (e.g., Claude 3.5 Sonnet, GPT-4o). For documents exceeding the model's context window, pre-chunk the document by major section boundaries, collapse each chunk independently, then run a final cross-chunk deduplication pass. Do not use tool-calling or function-calling modes for this prompt—it's a pure text-in, text-out transformation. Logging should capture: the original document hash, the collapsed document hash, the number of sections before and after collapse, validation pass/fail status, retry count, and any human review flags. Store both versions for auditability. If the document is destined for a regulated domain (legal, medical, financial), always route collapsed output through a human reviewer before publication, regardless of validation results.

Common failure modes to instrument for: the model merging sections that are thematically similar but factually distinct (false-positive collapse), the model dropping a section entirely (anchor loss), or the model rewriting content during merge rather than preserving original phrasing. Add eval checks that sample 10% of collapsed section pairs and compare key claims against the original sections. If claim preservation drops below 95%, increase the retry constraint strictness or escalate. For production systems, consider running this prompt as an asynchronous post-processing job rather than blocking the user-facing response—document generation latency is already high, and collapse adds meaningful processing time.