Prompt Assembly Regression Test Diff Prompt

This prompt is for prompt engineers and AI platform teams running pre-release validation on prompt assembly pipelines. Use it when you have changed your prompt template, variable substitution logic, context packing order, or tool schema injection and need to verify that the assembled prompt output matches expectations before deployment. The prompt compares the fully assembled prompt text produced by two versions of your assembly pipeline against a golden dataset of expected outputs. It produces a structured diff showing added, removed, and modified sections, classifies each change as expected or unexpected, and emits a pass/fail signal suitable for CI/CD release gates.

The ideal user has already built a prompt assembly pipeline that combines templates, variables, retrieved context, tool schemas, and few-shot examples into a final model request. You have a golden dataset of known inputs and their expected assembled outputs. You are about to deploy a change to the assembly logic and need an automated gate that catches regressions before they reach production. This prompt fits into a pre-release QA workflow where it receives two assembled prompt variants and the expected output, then produces a structured comparison report that your CI/CD system can parse for a go/no-go decision.

Do not use this prompt for comparing raw prompt templates without assembly, for evaluating model output quality, or for runtime production monitoring. It is not designed to judge whether a model response is correct—only whether the assembled prompt text matches expectations. It is also not a replacement for behavioral eval suites that measure end-to-end task accuracy. This prompt belongs in your pre-release QA pipeline, not in your inference path. If you need runtime observability, use the Prompt Assembly Trace ID Injection Template instead. If you need to compare two prompt versions without a golden dataset, use the Assembled Prompt Diff for Version Comparison.

This prompt template is designed to be pasted directly into your regression test harness. It instructs the model to act as a prompt assembly diff engine, comparing the output of two prompt versions—[CURRENT_VERSION] and [PREVIOUS_VERSION]—against a set of golden test cases. The model will produce a structured diff report that identifies which sections of the assembled prompt changed, whether those changes were expected, and whether the output meets the pass/fail criteria defined in your release gate policy. Replace every square-bracket placeholder with your actual values before running the test.

codeCopy
You are a prompt assembly regression test diff engine. Your task is to compare the assembled prompt output from two versions against a golden dataset and produce a structured diff report.

## INPUTS
- Current Version Output: [CURRENT_VERSION_OUTPUT]
- Previous Version Output: [PREVIOUS_VERSION_OUTPUT]
- Golden Test Cases: [GOLDEN_TEST_CASES]
- Expected Changes Manifest: [EXPECTED_CHANGES_MANIFEST]
- Release Gate Policy: [RELEASE_GATE_POLICY]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "diff_summary": {
    "total_sections_compared": <integer>,
    "sections_changed": <integer>,
    "sections_unchanged": <integer>,
    "unexpected_changes_detected": <boolean>,
    "release_gate_result": "PASS" | "FAIL" | "REVIEW_REQUIRED"
  },
  "section_diffs": [
    {
      "section_name": "<string>",
      "change_type": "ADDED" | "REMOVED" | "MODIFIED" | "UNCHANGED",
      "expected": <boolean>,
      "before_snippet": "<string or null>",
      "after_snippet": "<string or null>",
      "impact_assessment": "LOW" | "MEDIUM" | "HIGH",
      "notes": "<string>"
    }
  ],
  "golden_test_results": [
    {
      "test_case_id": "<string>",
      "current_version_passed": <boolean>,
      "previous_version_passed": <boolean>,
      "regression_detected": <boolean>,
      "output_diff_summary": "<string>"
    }
  ],
  "recommendations": ["<string>"]
}

## CONSTRAINTS
- Compare section by section: system message, user input, tool schemas, few-shot examples, retrieved context, output format constraints, and any other injected blocks.
- Flag any section change that does not appear in the Expected Changes Manifest as unexpected.
- For each golden test case, evaluate whether the current version output still satisfies the acceptance criteria. A regression is when the current version fails a test that the previous version passed.
- If any unexpected change has HIGH impact, set release_gate_result to "FAIL".
- If unexpected changes exist but all are LOW impact, set release_gate_result to "REVIEW_REQUIRED".
- If all changes are expected and no regressions are detected, set release_gate_result to "PASS".
- Do not hallucinate section names. Use only the sections present in the provided outputs.
- If the inputs are malformed or missing required fields, return an error object: {"error": "<description>", "missing_fields": ["<field_name>"]}.

## RISK LEVEL
[RISK_LEVEL]

## EXAMPLES
[EXAMPLES]

After pasting this template, adapt it by replacing the placeholders with your actual test data. [CURRENT_VERSION_OUTPUT] and [PREVIOUS_VERSION_OUTPUT] should contain the fully assembled prompt strings or structured representations from each version. [GOLDEN_TEST_CASES] is your dataset of input-output pairs with acceptance criteria. [EXPECTED_CHANGES_MANIFEST] is a list of sections you intentionally modified between versions. [RELEASE_GATE_POLICY] defines your organization's rules for blocking or approving a release based on diff severity and regression count. [RISK_LEVEL] should be set to "HIGH" if this prompt assembly is used in regulated, safety-critical, or revenue-dependent workflows—this triggers stricter validation and human review requirements. [EXAMPLES] should include at least one passing and one failing diff scenario to calibrate the model's judgment. Before running this in CI/CD, validate the output against the JSON schema using a programmatic validator. If the model returns malformed JSON, implement a retry with the error message injected as additional context. For high-risk deployments, always route FAIL and REVIEW_REQUIRED results to a human reviewer before proceeding with the release.

This prompt is designed to run inside an automated regression testing harness, not as a one-off manual check. The harness should execute the diff prompt after every prompt assembly change, comparing the assembled output for a fixed golden dataset against a known-good baseline. The core loop is: assemble the prompt for each golden input using both the baseline and candidate versions, send both assembled payloads through the diff prompt, and evaluate whether the reported changes are expected or require a release gate to block deployment.

The harness must enforce several preflight checks before invoking the diff prompt. First, validate that both assembled prompts are structurally complete—no unresolved placeholders, no truncated sections, and no missing required blocks such as system instructions or tool schemas. Second, compute a content hash of each assembled prompt and store it alongside the diff result for auditability. Third, inject a trace ID and version stamp into the metadata of each assembled prompt so the diff output can be correlated back to the exact assembly configuration. The diff prompt itself should receive the two assembled prompts as [BASELINE_PROMPT] and [CANDIDATE_PROMPT], along with an [EXPECTED_CHANGES] manifest describing which sections were intentionally modified. The output must conform to a strict [OUTPUT_SCHEMA] that includes a change_blocks array, a verdict field set to PASS, WARN, or FAIL, and a release_gate_blocked boolean.

After the diff prompt returns, the harness must run a validation layer before accepting the result. Check that every change block references real sections present in both prompts—flag any hallucinated section names as a harness-level failure. Compare the reported changes against the [EXPECTED_CHANGES] manifest: unexpected additions or removals should escalate the verdict to FAIL and set release_gate_blocked: true. Log the full diff output, the harness validation result, and the content hashes to your observability stack. If the verdict is WARN or FAIL, route the diff report to a human review queue with a 24-hour SLA before the prompt change can proceed to production. For low-risk assembly changes where [RISK_LEVEL] is set to low, you may configure the harness to auto-approve PASS verdicts without human review, but never auto-approve a FAIL.

Model choice matters for this harness. The diff prompt requires strong instruction-following and structured output reliability, so prefer models with proven JSON mode or structured output support. Run the diff prompt with temperature set to 0 and seed fixed for reproducibility. Implement a retry policy with up to two retries if the output fails schema validation; if retries are exhausted, escalate to a human operator with the raw assembled prompts and the validation error. Do not silently swallow diff failures—an unparseable diff output is itself a release gate blocker because it means you cannot certify what changed between prompt versions.