Post-Release Validation Smoke Test Prompt

This prompt is designed for release engineers and SRE teams who need to define automated smoke test criteria immediately after a production deployment. It takes a service catalog, recent deployment details, and critical user journey definitions as input, and generates a prioritized checklist of validations. The output includes health checks, key transaction tests, integration point verifications, explicit pass/fail thresholds, and escalation triggers. Use this prompt when you need a structured, repeatable post-deployment verification artifact that can be fed into a CI/CD pipeline or an automated testing harness.

The prompt is most effective when you provide concrete, machine-readable context. Instead of a vague description like 'check if the payment service is working,' supply the actual health endpoint URL, the expected HTTP status code, and the maximum acceptable latency in milliseconds. The prompt will then generate a specific test case with a clear pass/fail condition, such as 'GET /payments/health must return 200 within 500ms.' This level of detail allows the generated checklist to be directly consumed by a synthetic monitoring tool or a custom test runner without manual interpretation. The prompt also requires you to define escalation triggers—for example, 'if the checkout flow fails for more than 2% of test runs, page the on-call engineer'—which transforms a static checklist into an operational runbook.

This prompt is not a substitute for real-time monitoring dashboards, anomaly detection systems, or incident response procedures. It is a planning and criteria-generation tool that produces a verification artifact before or immediately after deployment. Do not use it to diagnose an active incident, as it lacks access to live telemetry. The generated output should be reviewed by a human who understands the deployment's blast radius before it is loaded into an automated harness, especially for high-risk releases involving schema migrations or payment systems. After generating the checklist, the next step is to wire it into your CI/CD pipeline's post-deployment stage, ensuring each test case is executed and its result is logged for auditability.

This prompt template is designed to be pasted directly into your AI system or CI/CD pipeline agent. It instructs the model to act as a release engineer and generate a structured, prioritized smoke test checklist for a newly deployed service. The prompt uses square-bracket placeholders that you must replace with your specific deployment context—service name, health endpoints, critical user journeys, and monitoring tool configurations.

codeCopy
You are a senior release engineer responsible for post-deployment validation. A new version of [SERVICE_NAME] has just been deployed to the [ENVIRONMENT] environment. Your task is to generate a prioritized smoke test checklist to validate the deployment's health.

## Deployment Context
- **Service:** [SERVICE_NAME]
- **Version:** [NEW_VERSION_TAG]
- **Environment:** [ENVIRONMENT]
- **Deployment Time:** [DEPLOYMENT_TIMESTAMP]
- **Change Summary:** [BRIEF_DESCRIPTION_OF_CHANGES]

## Available Health Signals
- **Health Endpoints:** [LIST_OF_HEALTH_CHECK_URLS]
- **Key Transactions:** [LIST_OF_CRITICAL_USER_JOURNEYS_OR_API_PATHS]
- **Integration Points:** [LIST_OF_DEPENDENT_SERVICES_OR_EXTERNAL_APIS]
- **Monitoring Dashboards:** [LIST_OF_DASHBOARD_URLS_OR_METRIC_QUERIES]

## Output Schema
Generate a JSON object with the following structure:
{
  "service": "[SERVICE_NAME]",
  "version": "[NEW_VERSION_TAG]",
  "environment": "[ENVIRONMENT]",
  "generated_at": "[DEPLOYMENT_TIMESTAMP]",
  "smoke_tests": [
    {
      "priority": "P0-Critical | P1-High | P2-Medium",
      "category": "Health Check | Key Transaction | Integration | Observation",
      "test_name": "Short, descriptive name",
      "description": "What to validate and why it matters",
      "pass_criteria": "Specific, measurable condition for success (e.g., HTTP 200 within 2s)",
      "failure_threshold": "Condition that triggers a failure (e.g., 3 consecutive non-200 responses)",
      "escalation_trigger": "When to stop testing and escalate (e.g., Any P0 failure)",
      "suggested_tool_or_command": "curl, browser, monitoring query, or specific tool"
    }
  ],
  "overall_pass_condition": "Statement of what constitutes a passing smoke test run",
  "escalation_contact": "[ON_CALL_CONTACT_OR_CHANNEL]"
}

## Constraints
- Prioritize tests by blast radius: start with health checks, then key transactions, then integrations.
- Every test must have a measurable pass/fail criterion. Avoid subjective checks like "looks good."
- Include at least one observation period test that monitors error rates and latency for [OBSERVATION_WINDOW_MINUTES] minutes.
- If the change summary mentions database migrations, include a test for data integrity on critical tables.
- If the change summary mentions configuration changes, include a test that validates the new config values are live.

## Risk Level
[RISK_LEVEL: low | medium | high]
- If risk is high, double the observation period and add a rollback readiness check.

After pasting this template, replace every square-bracket placeholder with concrete values from your deployment pipeline. For example, [SERVICE_NAME] becomes payment-api, [ENVIRONMENT] becomes production-us-east-1, and [LIST_OF_HEALTH_CHECK_URLS] becomes a comma-separated list of actual endpoints. The [RISK_LEVEL] placeholder should be set dynamically based on your deployment risk scoring rubric. If your pipeline cannot provide a risk level, default to medium and add a note that a human should adjust the observation period. The generated JSON output is designed to be machine-readable so your CI/CD system can parse it and potentially execute the checks automatically. Always route the final checklist through a human release manager for approval before execution begins, especially for high-risk deployments.

The Post-Release Validation Smoke Test Prompt is designed to be called immediately after a deployment completes, as the first automated verification step before any traffic is routed to the new version. In a CI/CD harness, this prompt should be invoked by a post-deployment job that has access to the deployment manifest, the service health endpoints, and the list of critical transactions that must pass. The prompt itself does not execute checks—it generates a structured checklist that your automation framework then executes. This separation is critical: the model defines what to validate and why, while your pipeline owns the actual HTTP calls, metric queries, and threshold comparisons.

To integrate this into a pipeline, wrap the prompt call in a script that provides the required inputs: [DEPLOYMENT_MANIFEST] (the list of changed services, configs, and infrastructure), [CRITICAL_PATHS] (key user journeys and API contracts that must not break), [HEALTH_ENDPOINTS] (the URLs and expected responses for each service), and [OBSERVABILITY_DASHBOARDS] (links to error budgets, latency graphs, and business metrics). The model returns a JSON checklist with items, pass/fail thresholds, and escalation triggers. Your harness then iterates over this checklist, executes each validation, and compares results against the thresholds. Any failure that meets the escalation criteria should automatically page the on-call engineer and halt the canary or full rollout. Log every checklist item, the raw result, and the model's recommendation for auditability.

For retries and resilience, implement a two-tier approach. First, if the prompt call itself fails or returns malformed JSON, retry up to two times with exponential backoff before falling back to a static smoke test checklist maintained in your repository. Second, for individual checklist items that fail, allow a single re-check after a 30-second wait to filter out transient startup delays. If a re-check still fails, treat it as a confirmed failure. Model choice matters here: use a fast, instruction-following model like claude-3-5-sonnet or gpt-4o for checklist generation, not a reasoning-heavy model that adds latency. The prompt output should be validated against a strict JSON schema before execution begins—reject any checklist that omits required fields like check_id, threshold, or escalation_trigger. Finally, store the full prompt input, output, and execution results in your deployment audit log. This trace is essential for post-incident review and for calibrating whether the prompt's risk assessment matches real production behavior over time.