Model Update Rollout Confirmation Prompt

This prompt is for MLOps and platform engineering teams managing AI model deployments. Use it when you need a structured, human-readable summary of a planned model update before any production traffic is shifted. The prompt forces the system to compare evaluation metrics between the current and candidate model, confirm shadow-mode results, specify the traffic shift percentage, and define explicit rollback criteria. It belongs in a deployment pipeline gate where a human must review and approve the rollout before the canary or blue-green deployment proceeds.

Do not use this prompt for infrastructure provisioning, code merges, or database migrations. It is purpose-built for model version transitions where evaluation evidence and rollback safety are the primary concerns. The prompt assumes you already have a candidate model registered in a model registry, a current production model serving traffic, and evaluation results from both offline test sets and shadow-mode inference. If you lack shadow-mode comparison data or a defined traffic-shift mechanism, this prompt will surface those gaps rather than paper over them.

Wire this prompt into your deployment pipeline immediately after shadow-mode evaluation completes and before any kubectl apply, aws sagemaker update-endpoint, or equivalent production mutation. The output should block automated promotion and require a human to read the summary, confirm the evidence, and approve the rollout. If your team practices continuous deployment with automated canary analysis, use this prompt as the final manual gate for high-risk models or regulated environments where an auditor must see a signed confirmation record.

The prompt below is designed to be injected into an MLOps deployment pipeline at the approval gate. It forces the model to produce a structured, human-readable summary of the planned model update, including evaluation metric comparisons, traffic shift percentages, and explicit rollback criteria. The output is intended to be reviewed by a release manager or SRE before the canary or blue-green deployment proceeds. Do not use this prompt for fully automated, zero-human-touch rollouts; it is a checkpoint, not a replacement for a robust CI/CD pipeline.

codeCopy
You are an MLOps release reviewer. Your task is to generate a concise, structured summary of a planned model update rollout. You will receive details about the current model, the candidate model, evaluation results, and the rollout plan. Your output must be a JSON object that a human operator can review before approving the deployment.

## Inputs
- [CURRENT_MODEL_VERSION]: The model version currently serving production traffic.
- [CANDIDATE_MODEL_VERSION]: The new model version proposed for rollout.
- [EVALUATION_REPORT]: A structured object containing metric comparisons between the current and candidate models. Include fields for metric name, current value, candidate value, threshold, and pass/fail status.
- [TRAFFIC_SHIFT_PERCENTAGE]: The percentage of production traffic to route to the candidate model.
- [ROLLBACK_CRITERIA]: A list of conditions that will trigger an automatic rollback (e.g., error rate > X%, latency p99 > Y ms).
- [SHADOW_MODE_RESULTS]: Optional. Results from a shadow-mode deployment comparing the candidate model's predictions against production outcomes. May be null.
- [DEPLOYMENT_WINDOW]: The scheduled start and end time for the rollout.

## Output Schema
Generate a JSON object with the following structure:
{
  "rollout_summary": {
    "current_model": "string",
    "candidate_model": "string",
    "traffic_shift_percentage": "number",
    "deployment_window": "string",
    "risk_level": "low | medium | high",
    "risk_rationale": "string"
  },
  "evaluation_comparison": [
    {
      "metric": "string",
      "current_value": "number",
      "candidate_value": "number",
      "threshold": "number",
      "status": "pass | fail | no_threshold",
      "delta_explanation": "string"
    }
  ],
  "shadow_mode_validation": {
    "available": "boolean",
    "summary": "string | null",
    "concerns": ["string"]
  },
  "rollback_plan": {
    "triggers": ["string"],
    "rollback_procedure": "string"
  },
  "required_approvals": ["string"],
  "missing_information": ["string"]
}

## Constraints
- If [SHADOW_MODE_RESULTS] is null or unavailable, set `shadow_mode_validation.available` to false and add a specific concern to `missing_information` explaining the risk of proceeding without shadow data.
- If any evaluation metric has a `fail` status, set `risk_level` to "high" and explain the failing metric in `risk_rationale`.
- If [TRAFFIC_SHIFT_PERCENTAGE] is greater than 10%, set `risk_level` to at least "medium" and justify the percentage in `risk_rationale`.
- The `rollback_procedure` must reference the specific [CURRENT_MODEL_VERSION] to revert to.
- Do not fabricate evaluation data. Only use the provided [EVALUATION_REPORT].
- If any required input is missing, list it in `missing_information` and set `risk_level` to "high".

Adaptation notes: Replace each square-bracket placeholder with values from your model registry and evaluation store. The [EVALUATION_REPORT] should be a serialized JSON object matching the expected structure. If your deployment system uses a different rollout strategy (e.g., canary stages instead of a single traffic shift), modify the traffic_shift_percentage field to reflect your actual stages. The required_approvals field should be populated based on your organization's release policy. Before wiring this into an automated gate, run the prompt against at least five historical rollouts (both successful and rolled-back) to validate that the risk level and missing information flags align with your team's judgment.

This prompt is designed to act as a pre-release gate in a CI/CD pipeline for model deployments. It should be invoked after shadow-mode evaluation completes but before production traffic is shifted. The harness must supply structured inputs—model registry metadata, evaluation metric comparisons, traffic shift percentage, and rollback criteria—and then parse the generated summary to drive an automated approval workflow. The output is not just a human-readable summary; it is a machine-readable checkpoint that downstream systems can use to block or allow the rollout.

The implementation should wrap the prompt in a deployment step that validates the output against a strict schema. Use a JSON mode or structured output API to enforce fields such as model_version, metric_delta, traffic_percentage, rollback_conditions, and shadow_evaluation_status. After generation, run a validation script that checks for missing shadow-mode results, incomplete metric comparisons, or a traffic shift that exceeds pre-configured thresholds. If validation fails, the pipeline must halt and surface the specific failure to the MLOps team. For high-risk deployments, integrate a human-in-the-loop step: post the generated summary to a review queue (e.g., Slack, Jira, or a custom approval UI) and require explicit sign-off before the canary deployment proceeds.

Log every invocation, including the full prompt payload, the generated summary, validation results, and the reviewer's decision, to an audit store. This creates an evidence trail for governance reviews. Avoid using this prompt for rollbacks or emergency hotfixes where speed is critical; those workflows should bypass the confirmation gate but still log the action. Next, connect this harness to your model registry's webhook so that any new model version tagged for production automatically triggers the summary generation and approval flow.