Migration Success Verification Prompt for QA Pipelines

This prompt is designed for QA engineers and platform teams who have already migrated active AI sessions to new instruction versions and need an automated gate to validate the results. The core job-to-be-done is comparing pre-migration and post-migration outputs against a set of defined acceptance criteria and flagging any regressions that should block the rollout. It assumes you have recorded pre-migration behavior, a batch of migrated sessions, and a clear set of criteria that define what 'equivalent or better' means for your use case. Without these prerequisites, the prompt cannot produce a reliable verification report.

The ideal user is a QA engineer or release manager running this prompt inside a CI/CD pipeline's QA gate, triggered after a session migration dry-run or batch migration job completes. Required context includes the pre-migration session transcript, the post-migration session transcript, the acceptance criteria document, and any domain-specific rules that define acceptable behavioral variance. Do not use this prompt for initial prompt design validation, for testing sessions that were never migrated, or for evaluating raw model outputs without a migration context. It is specifically scoped to regression detection across an instruction version boundary.

In practice, you will wire this prompt into an automated test harness that feeds paired session transcripts and acceptance criteria, then parses the structured verification report. The report should include a pass/fail determination per session, a regression severity classification for any failures, and a summary of which acceptance criteria were violated. Common failure modes include false positives from acceptable behavioral variance that wasn't captured in the criteria, and false negatives from criteria that are too permissive. To mitigate these, maintain a living set of acceptance criteria that evolves with your instruction changes, and always include a human review step for any session flagged as a regression before blocking a production rollout.

This template is the core of the Migration Success Verification Prompt for QA Pipelines. It is designed to be copied directly into your testing harness, an automated eval runner, or a manual QA review interface. The prompt instructs the model to act as a QA engineer, comparing pre- and post-migration behavior for a given session against a set of acceptance criteria. The output is a structured verification report that flags regressions, confirms expected changes, and notes any ambiguous behavior.

textCopy
You are a QA engineer validating a session migration. Your task is to compare the behavior of an AI agent before and after an instruction update to ensure the migration was successful.

You will be given the following inputs:
- [PRE_MIGRATION_TRANSCRIPT]: The full conversation transcript before the instruction update.
- [POST_MIGRATION_TRANSCRIPT]: The full conversation transcript after the instruction update, starting from the same user context.
- [ACCEPTANCE_CRITERIA]: A list of specific, verifiable behavioral requirements that the post-migration session must meet.
- [MIGRATION_CONTEXT]: A description of what changed in the instructions and why.

Your task is to produce a Verification Report in the following JSON format:
{
  "migration_id": "string",
  "overall_status": "PASS" | "FAIL" | "INCONCLUSIVE",
  "criteria_results": [
    {
      "criterion_id": "string",
      "description": "string",
      "status": "MET" | "NOT_MET" | "UNABLE_TO_VERIFY",
      "evidence": "A direct quote or specific reference from the transcripts.",
      "rationale": "A brief explanation of why the status was assigned."
    }
  ],
  "regressions_found": [
    {
      "description": "string",
      "pre_migration_evidence": "string",
      "post_migration_evidence": "string",
      "severity": "CRITICAL" | "HIGH" | "MEDIUM" | "LOW"
    }
  ],
  "unexpected_changes": [
    "A description of any behavioral change not covered by the acceptance criteria."
  ],
  "summary": "A concise 2-3 sentence summary of the migration outcome."
}

Follow these constraints:
- [CONSTRAINTS]: Base all findings strictly on the provided transcripts. Do not infer intent. If evidence is insufficient, use "UNABLE_TO_VERIFY". A regression is any behavior that was correct pre-migration but is incorrect or missing post-migration.

To adapt this template, replace the square-bracket placeholders with your test data. For automated pipelines, [PRE_MIGRATION_TRANSCRIPT] and [POST_MIGRATION_TRANSCRIPT] can be loaded from your test case database. The [ACCEPTANCE_CRITERIA] should be a structured list derived from your migration plan's success metrics. The [CONSTRAINTS] placeholder can be extended with domain-specific rules, such as requiring citations to specific document sections or flagging tone violations. After running this prompt, always validate the output JSON against your expected schema before marking a migration as verified. For high-risk migrations, the output of this prompt should be a mandatory input to a human approval gate, not an automated pass/fail decision.

The Migration Success Verification Prompt is designed to be a gate in your CI/CD pipeline, not a one-off manual check. It should run automatically after every instruction version bump or session migration event. The prompt expects a structured input payload containing pre-migration and post-migration session traces, the acceptance criteria for the migration, and the version identifiers for both the old and new instruction sets. The output is a structured verification report that your pipeline can parse to decide whether to promote the migration, flag regressions, or block the rollout.

To integrate this prompt, wrap it in a harness that first collects the required inputs: [PRE_MIGRATION_TRACE], [POST_MIGRATION_TRACE], [ACCEPTANCE_CRITERIA], [OLD_INSTRUCTION_VERSION], and [NEW_INSTRUCTION_VERSION]. The harness should validate that these inputs are non-empty and that the version identifiers match the expected migration pair. After calling the model, parse the [OUTPUT_SCHEMA]—a JSON object containing a verification_status (PASS, FAIL, or INCONCLUSIVE), a list of regression_flags, and a behavioral_comparison_summary. Implement a retry loop with a maximum of 2 retries if the model returns malformed JSON or fails to include the required fields. Log every attempt, including the raw model response and the parsed output, for auditability. For high-risk migrations in regulated environments, add a human approval step: if verification_status is not PASS, or if any regression_flags have a severity of HIGH, block the pipeline and route the report to a review queue before proceeding.

Choose a model with strong instruction-following and structured output capabilities for this task. The prompt relies on precise comparison and schema adherence, so smaller or less capable models may produce inconsistent verification reports. If you are running this in a CI/CD environment with cost constraints, consider caching the prompt prefix that contains the static instructions and output schema. The dynamic inputs—the traces and criteria—should be appended after the cached prefix. Before deploying this harness to production, build a golden test set of known migration scenarios: one where the migration should PASS, one where it should FAIL due to a behavioral regression, and one where the session state was corrupted. Run these through the harness and assert that the verification_status matches the expected outcome for each case. This pre-release eval gate will catch prompt brittleness before it affects a real rollout.