Assembled Prompt Diff for Version Comparison

This playbook is for prompt engineers and AI operations teams who need to compare two versions of an assembled prompt side by side before releasing a change. The prompt produces a structured diff showing added, removed, and modified sections across instruction blocks, few-shot examples, tool schemas, and output constraints. Use it as a pre-release gate to flag unexpected changes, verify that only intended sections were modified, and generate regression test evidence. This prompt belongs in your CI/CD pipeline for prompts, right after assembly and before deployment. It assumes you already have two fully assembled prompt payloads ready for comparison.

The ideal user is an engineering lead or prompt engineer who owns a prompt assembly pipeline and needs automated verification that a template change, variable substitution update, or context source modification produced only the intended differences. You should have a golden dataset of expected inputs and a known-good reference version of the assembled prompt. The diff output should be machine-readable so your CI/CD system can block deployment if unexpected sections changed, if token counts shifted beyond a threshold, or if critical instruction blocks were accidentally removed. Wire the diff into your regression test suite alongside your existing prompt assembly validation checks.

Do not use this prompt for real-time production monitoring or for comparing raw model outputs. It is designed for pre-release comparison of assembled prompt payloads, not for evaluating response quality or detecting drift in model behavior. If you need to compare model responses across versions, use an LLM judge or evaluation rubric prompt instead. For runtime observability, pair this with the Prompt Assembly Trace ID Injection Template to correlate diffs with production request traces. After running the diff, review any unexpected changes manually before approving the release, especially for prompts used in regulated or high-risk domains where instruction changes could alter refusal behavior or compliance posture.

This template is designed to compare two fully assembled prompt payloads—not just the raw templates, but the rendered output including all injected context, tool schemas, examples, and instructions. Replace the square-bracket placeholders with your actual prompt versions, configuration, and evaluation criteria. The prompt instructs the model to produce a structured diff that highlights additions, removals, and modifications across every section of the assembled prompt, making it suitable for pre-release regression gates and incident diagnosis.

textCopy
You are a prompt assembly auditor. Your task is to compare two versions of a fully assembled prompt payload and produce a structured diff.

## INPUT
[PROMPT_VERSION_A]
[PROMPT_VERSION_B]

## CONFIGURATION
- Assembly pipeline version: [ASSEMBLY_PIPELINE_VERSION]
- Model target: [MODEL_IDENTIFIER]
- Sections to compare: [SECTION_LIST] (e.g., system_message, user_input, retrieved_context, tool_schemas, few_shot_examples, output_schema, policy_constraints)
- Ignore sections: [IGNORE_SECTION_LIST]
- Diff granularity: [DIFF_GRANULARITY] (options: section_level, paragraph_level, sentence_level)

## OUTPUT_SCHEMA
Return a JSON object with the following structure:
{
  "diff_summary": {
    "version_a_hash": "string",
    "version_b_hash": "string",
    "total_sections_compared": integer,
    "sections_unchanged": integer,
    "sections_modified": integer,
    "sections_added": integer,
    "sections_removed": integer,
    "breaking_changes_detected": boolean,
    "breaking_change_details": ["string"]
  },
  "section_diffs": [
    {
      "section_name": "string",
      "change_type": "unchanged | modified | added | removed",
      "before": "string | null",
      "after": "string | null",
      "unified_diff": "string",
      "semantic_change_summary": "string",
      "risk_level": "low | medium | high",
      "risk_rationale": "string"
    }
  ],
  "regression_flags": [
    {
      "flag_id": "string",
      "section": "string",
      "description": "string",
      "expected_behavior": "string",
      "observed_change": "string",
      "severity": "warning | blocking"
    }
  ]
}

## CONSTRAINTS
- Compare only the sections listed in [SECTION_LIST]. Do not compare sections in [IGNORE_SECTION_LIST].
- For each modified section, produce a unified diff showing exact line-level changes.
- Flag any change that alters instruction priority, removes safety constraints, modifies tool argument schemas, or changes output format requirements as a potential breaking change.
- If [REGRESSION_TEST_CASES] are provided, check each modified section against expected behavior and populate the regression_flags array.
- Do not hallucinate differences. If two sections are semantically identical but formatted differently, note the formatting change but mark the section as unchanged in the summary count.
- If either version contains PII or secrets, redact them in the output and add a redaction notice to the diff_summary.

## REGRESSION TEST CASES
[REGRESSION_TEST_CASES]

## EXAMPLES
[DIFF_EXAMPLES]

After pasting the template, replace each placeholder with concrete values. For [PROMPT_VERSION_A] and [PROMPT_VERSION_B], insert the fully assembled prompt strings—these should be the exact payloads sent to the model, not the templates before variable substitution. Set [SECTION_LIST] to the specific sections you want compared, such as ["system_message", "tool_schemas", "output_schema"]. If you have a golden dataset of expected behaviors, populate [REGRESSION_TEST_CASES] with assertions like "system_message must contain refusal policy" or "tool_schemas must include required argument 'user_id'". The [DIFF_EXAMPLES] field should contain one or two example diffs that demonstrate your expected output format and risk classification standards. Before deploying this prompt into a CI/CD pipeline, validate the output against your schema, run it against known version pairs to calibrate risk thresholds, and ensure a human reviews any diff flagged with breaking_changes_detected: true before the new prompt version reaches production.

The Assembled Prompt Diff prompt is not a standalone tool—it is a comparison step inside a broader prompt versioning and release pipeline. In production, this prompt should be invoked automatically whenever a new prompt version is staged for deployment. The harness must supply two fully assembled prompt payloads: a baseline version (typically the current production prompt) and a candidate version (the proposed update). These payloads should be the actual rendered prompts after variable substitution, context injection, and tool schema assembly—not the raw templates. Comparing rendered outputs catches regressions that template-level diffs miss, such as incorrect variable bindings, broken conditional branches, or unintended context ordering changes.

The implementation should wrap the diff prompt in a preflight validation layer. Before calling the model, validate that both [BASELINE_PROMPT] and [CANDIDATE_PROMPT] are non-empty, that their version stamps or hashes differ, and that they conform to expected section boundaries. After receiving the structured diff output, run automated checks: confirm the diff schema includes added, removed, and modified arrays with section-level granularity; verify that instruction blocks, few-shot examples, and tool schemas are compared separately; and flag any unexpected changes against an allowlist of known-intentional modifications. For high-risk prompt changes—such as system message rewrites, safety policy updates, or tool schema modifications—route the diff output to a human reviewer before the release gate opens. Log the full diff payload alongside the version hashes, timestamp, and reviewer decision for auditability.

Integrate this diff step into your CI/CD pipeline for prompts. After the diff is generated and validated, use the regression_test_hooks field from the output to trigger targeted eval runs against a golden dataset. If the diff reveals changes to instruction blocks, run instruction-following evals. If tool schemas changed, run tool-selection accuracy tests. If few-shot examples were added or removed, run output-format consistency checks. Block deployment if any regression test fails or if the diff contains unexpected_changes that lack reviewer approval. Avoid running this prompt on every minor edit—reserve it for staged version comparisons where the cost of a detailed diff is justified by the risk of the change.