Automated Promotion Gate Decision Prompt Template

This prompt is the final decision synthesis step in a CI/CD pipeline for prompts. It is designed for release managers and MLOps engineers who need a single, auditable, and deterministic output—promote, hold, or rollback—derived from quantitative evidence. The ideal user is someone who has already executed a full suite of automated checks, including regression test suites, semantic drift detection, and canary comparison reports. The prompt consumes the structured results of those upstream processes, along with predefined gate thresholds, and produces a decision with a confidence score and a traceable evidence trail. It is not a diagnostic tool for individual test failures or a root cause analysis engine; it assumes the input data is complete and the upstream evaluations are trustworthy.

To use this prompt effectively, you must provide it with a structured payload containing three critical inputs: the evaluation run results (pass/fail counts, metric scores, and failure details), the canary comparison report (traffic split percentages, metric deltas, and statistical significance flags), and the predefined gate thresholds (minimum pass rates, maximum allowable drift, and latency budgets). The prompt is engineered to be deterministic, meaning it applies the rules you define to the data you provide without improvisation. It will output a structured decision object that includes the recommended action, a confidence score, a list of evidence that informed the decision, and any override justifications if a human has intervened. This makes it suitable for direct consumption by an automated deployment system or a human reviewer in a high-stakes environment.

Do not use this prompt for exploratory analysis, debugging flaky evaluations, or generating new test cases. It is a gate, not a workshop. If the input data is incomplete, the prompt will flag missing evidence rather than hallucinate a decision. For high-risk domains, the output should be routed to a human approval queue before any automated action is taken. The next step after receiving the output is to log the decision and evidence in an audit trail, then trigger the corresponding deployment action in your orchestration system.

This prompt template is designed to be the decision engine in your automated promotion gate. It consumes structured evaluation results, canary comparison reports, and your predefined gate thresholds to produce a consistent, evidence-backed promote/hold/rollback decision. The template uses square-bracket placeholders that your CI/CD pipeline should populate with actual data before sending the prompt to the model. Every placeholder maps to a specific artifact your pipeline already produces: eval run summaries, canary diff reports, threshold configurations, and override justifications. The output is a structured JSON decision object that downstream automation can act on without human parsing.

textCopy
You are an automated release gate decision engine for AI prompt deployments. Your job is to analyze evaluation results, canary comparison data, and predefined thresholds to produce a structured promote, hold, or rollback decision with supporting evidence.

## INPUT DATA

### Evaluation Run Results
[EVAL_RUN_RESULTS]

### Canary Comparison Report
[CANARY_COMPARISON_REPORT]

### Gate Threshold Configuration
[GATE_THRESHOLD_CONFIG]

### Override Justification (if any)
[OVERRIDE_JUSTIFICATION]

### Deployment Context
[DEPLOYMENT_CONTEXT]

## DECISION RULES

1. If any critical threshold is breached, the decision must be ROLLBACK.
2. If all critical thresholds pass but one or more warning thresholds are breached, the decision must be HOLD with specific concerns listed.
3. If all thresholds pass and no warnings are triggered, the decision must be PROMOTE.
4. If an override justification is provided, evaluate whether it addresses the specific failures with compensating controls. If it does not, reject the override and maintain the original decision.
5. If the canary comparison shows statistically significant regression on any metric, treat this as a critical threshold breach.
6. If eval run coverage is below the minimum required sample size, the decision must be HOLD regardless of pass/fail status.

## OUTPUT SCHEMA

Return a JSON object with this exact structure:
{
  "decision": "PROMOTE|HOLD|ROLLBACK",
  "confidence": 0.0-1.0,
  "summary": "One-sentence explanation of the decision",
  "threshold_results": [
    {
      "threshold_name": "string",
      "threshold_type": "critical|warning",
      "expected_value": "number or string",
      "actual_value": "number or string",
      "passed": true/false,
      "evidence": "specific data point from eval or canary report"
    }
  ],
  "regressions_detected": [
    {
      "metric": "string",
      "baseline_value": "number",
      "canary_value": "number",
      "p_value": "number or null",
      "severity": "low|medium|high|critical"
    }
  ],
  "override_evaluation": {
    "override_requested": true/false,
    "override_accepted": true/false/null,
    "reasoning": "string or null"
  },
  "unresolved_concerns": ["list of specific issues that remain unresolved"],
  "recommended_actions": ["concrete next steps based on the decision"],
  "evidence_citations": ["specific references to sections or data points in the input reports"]
}

## CONSTRAINTS

- Base every threshold_result on explicit data from the input reports. Do not infer or assume values.
- If the canary comparison report includes p-values or confidence intervals, use them to determine statistical significance. If not provided, note this as a limitation in unresolved_concerns.
- When recommending HOLD, specify exactly which thresholds must improve and by how much before promotion can proceed.
- When recommending ROLLBACK, cite the specific critical failure that triggered the decision.
- If the override justification is empty or missing, set override_requested to false and override_accepted to null.
- Do not fabricate evidence citations. Each citation must reference a real section, metric name, or data point present in the input reports.
- If input data is malformed or missing required fields, set decision to HOLD and list the missing fields in unresolved_concerns.

To adapt this template for your pipeline, replace each placeholder with the actual data your CI/CD system generates. The [EVAL_RUN_RESULTS] placeholder should receive the full output of your evaluation harness, including per-test-case results, aggregate scores, and coverage statistics. The [CANARY_COMPARISON_REPORT] placeholder expects a structured diff between baseline and canary outputs, ideally with statistical significance measures. The [GATE_THRESHOLD_CONFIG] placeholder should contain your organization's specific pass/fail criteria, including which thresholds are critical versus warning-level. The [OVERRIDE_JUSTIFICATION] placeholder should only be populated when a human has explicitly requested a gate bypass, and it must include the reason, risk assessment, and compensating controls. The [DEPLOYMENT_CONTEXT] placeholder should carry metadata about the environment, prompt version, and affected workflows. After populating these placeholders, validate the output JSON against the schema before allowing any automated action. For high-risk deployments, route the structured decision to a human approval queue rather than acting on it automatically, even when the model returns PROMOTE.

This prompt is designed to be the final decision step in an automated prompt release pipeline. It should be invoked only after all upstream gates—smoke tests, canary comparisons, eval suites, and statistical significance checks—have completed and produced structured output artifacts. The prompt consumes those artifacts as [INPUT] and applies the predefined gate thresholds from [CONSTRAINTS] to produce a machine-readable promote/hold/rollback decision. Do not call this prompt in isolation without the required eval run results and threshold definitions; doing so will produce a decision based on hallucinated evidence rather than real pipeline data.

Wire this prompt into your CI/CD system as a dedicated gate stage that runs after all eval jobs complete. The implementation should: (1) aggregate eval run results, canary comparison reports, and any override flags into a single structured JSON payload for the [INPUT] placeholder; (2) inject the organization's gate threshold policy—minimum pass rates, maximum regression counts, required statistical significance levels—into the [CONSTRAINTS] placeholder; (3) call the model with a low temperature setting (0.0–0.1) to maximize decision consistency; (4) validate the output against the expected [OUTPUT_SCHEMA] before accepting it; and (5) log the full prompt, response, and validation result to your artifact store for auditability. If the output fails schema validation, retry once with the validation error appended as context. If the retry also fails, escalate to a human release manager rather than silently defaulting to hold or promote.

For high-risk deployments, implement a mandatory human review step when the model outputs decision: "hold" or when the confidence score falls below a configured threshold. The override justification field in the output schema allows a human to document why a hold decision was overridden to promote, or vice versa. Store this justification alongside the automated decision in your deployment audit trail. Avoid wiring this prompt directly to an automated deployment action without a circuit breaker: if the model returns an unparseable response or the validation layer catches a schema violation, the pipeline must halt and alert, never silently proceed. For model choice, use a model with strong instruction-following and structured output capabilities. If your pipeline already uses a specific model for eval judging, prefer the same model family for decision consistency.