Canary Deployment Analysis Failure Retry Prompt Template

This prompt is for release engineers and SRE teams who need an automated, evidence-based decision when a canary deployment fails its health or performance metrics. The job-to-be-done is not just to retry the deployment, but to analyze the specific metric failure, compare it against the stable baseline, assess the error budget impact, and produce a clear, auditable recommendation: rollback, pause for human review, or adjust the progressive rollout parameters and retry. The ideal user is an engineer integrating this prompt into a CI/CD pipeline or an internal platform, where the prompt acts as the reasoning layer between metric collection and the deployment orchestrator.

You should use this prompt when your deployment system has already collected quantitative canary metrics (latency, error rate, saturation) and baseline data, and the canary has breached a pre-defined threshold. The prompt requires structured input: the specific metric that failed, its value, the baseline value, the error budget remaining, the current rollout percentage, and the deployment history. Do not use this prompt for initial deployment decisions, for infrastructure failures unrelated to the canary's own behavior (like a network partition), or when the metric data is incomplete. In those cases, a different retry or fallback prompt is required.

The primary constraint is blast radius limitation. The prompt's reasoning must prioritize preventing a bad change from reaching more users over the speed of the rollout. It must also be aware of false positive detection, distinguishing a transient spike from a sustained degradation. The next step after using this prompt is to feed its structured decision into your deployment orchestrator. Avoid using the raw text output directly for automated rollbacks; instead, parse the structured decision and have your harness enforce a hard limit on the number of retries and the maximum rollout percentage to prevent an infinite loop of failing canary analyses.

This prompt template is designed to be injected into your retry harness when a canary deployment's health metrics breach their defined thresholds. It forces the model to act as a release engineer, grounding its decision in provided metric data, error budgets, and blast radius constraints. The template uses square-bracket placeholders for all dynamic inputs, ensuring you can wire it directly into your deployment pipeline's decision node.

textCopy
You are a release engineer analyzing a failed canary deployment. Your task is to produce a structured decision: either `ROLLBACK`, `RETRY_WITH_ADJUSTMENTS`, or `ESCALATE`. Base your decision strictly on the provided context.

## DEPLOYMENT CONTEXT
- Service: [SERVICE_NAME]
- Canary Version: [CANARY_VERSION]
- Stable Version: [STABLE_VERSION]
- Canary Traffic Percentage: [CANARY_TRAFFIC_PERCENT]%
- Deployment Start Time: [DEPLOYMENT_START_TIME]
- Current Time: [CURRENT_TIME]

## METRIC COMPARISON
Compare the canary metrics against the stable baseline over the evaluation window.

[STRUCTURED_METRIC_COMPARISON]

## ERROR BUDGET
- Remaining Error Budget: [REMAINING_ERROR_BUDGET]%
- Error Budget Burn Rate (current window): [BURN_RATE]x
- SLO Target: [SLO_TARGET]%

## CONSTRAINTS
- Max Blast Radius: [MAX_BLAST_RADIUS]% of users
- Rollback Safety: [ROLLBACK_SAFETY_CHECK] (e.g., "No schema migrations in canary")
- Previous Retry Count for this version: [RETRY_COUNT]

## OUTPUT_SCHEMA
Respond with a single JSON object. Do not include any text outside the JSON.
{
  "decision": "ROLLBACK" | "RETRY_WITH_ADJUSTMENTS" | "ESCALATE",
  "confidence": 0.0-1.0,
  "reasoning": "A concise, evidence-based explanation referencing specific metrics and error budget impact.",
  "adjusted_traffic_percentage": null | number, // Only for RETRY_WITH_ADJUSTMENTS
  "escalation_reason": null | string, // Only for ESCALATE
  "false_positive_risk": "LOW" | "MEDIUM" | "HIGH"
}

## DECISION RULES
1. If the error budget burn rate exceeds 5x or remaining budget is below 10%, the decision must be `ROLLBACK`.
2. If a metric regression is detected but the blast radius is below the maximum and the error budget is healthy, consider `RETRY_WITH_ADJUSTMENTS` with a reduced traffic percentage.
3. If the failure pattern is intermittent, the metric comparison is inconclusive, or the rollback safety check fails, the decision must be `ESCALATE`.
4. A `false_positive_risk` of `HIGH` must be set if the metric deviation is within 2 standard deviations of the baseline.

To adapt this template, replace each square-bracket placeholder with data from your monitoring stack and deployment orchestrator. The [STRUCTURED_METRIC_COMPARISON] placeholder should be populated with a formatted text block—such as a markdown table or a list of key-value pairs—that explicitly compares latency, error rate, and saturation metrics between the canary and stable versions. Before putting this into production, run it against a golden dataset of known deployment outcomes to calibrate the confidence and false_positive_risk fields. Always log the full prompt and the model's JSON response for post-incident review.

The canary analysis retry prompt is designed to be called by a release orchestrator or CI/CD pipeline when an initial canary health check fails. The harness should invoke the prompt with structured input containing the failed metric payload, the canary-baseline comparison, the current error budget status, and the progressive rollout stage. The model's output is a structured decision object—retry_with_adjustment, rollback, or escalate—along with a confidence score and a concise rationale. This decision must be parsed and acted upon programmatically, not treated as a free-text suggestion.

The implementation must enforce a retry budget to prevent the prompt from looping indefinitely on ambiguous failures. A recommended starting point is a maximum of 3 prompt invocations per canary stage. Each retry must pass the previous decision and any new metric data as additional context. The harness should validate the output schema strictly: the decision field must be one of the three allowed enum values, the confidence field must be a float between 0.0 and 1.0, and the rationale must be a non-empty string. If validation fails, the harness should fall back to a conservative escalate action and log the malformed response for debugging. For high-risk production environments, add a human approval gate when the model recommends retry_with_adjustment with a confidence below 0.85 or when the error budget remaining is below 10%.

Model choice matters here. Use a model with strong structured output support (e.g., GPT-4o, Claude 3.5 Sonnet) and enable JSON mode or tool-calling with a strict schema. Do not use a small or general-purpose model for this task, as misclassifying a rollback decision can directly cause a production outage. Log every prompt invocation, the raw response, the parsed decision, and the harness action taken. This audit trail is essential for post-incident review and for tuning the retry budget thresholds over time. Avoid wiring this prompt directly to an automated rollback without a confirmation step unless your error budget policy explicitly permits fully autonomous rollback within defined blast-radius limits.