Incident Remediation Verification Prompt

This prompt is a post-action verification tool for incident responders and SRE teams. Its job is to close the loop on remediation by producing a structured report that confirms whether a deployed fix actually resolved the underlying incident. It is designed for the moment after a fix is live—when monitoring data, logs, and incident context are available—and the team needs a disciplined, evidence-based decision to close the incident or escalate further. The primary user is the incident commander or a designated responder who owns the technical verification step before an incident is formally resolved.

To use this prompt effectively, you must provide three categories of input: the incident context (a summary of the root cause and the fix applied), post-fix monitoring data (metrics, logs, and alerts from the affected systems), and verification criteria (the specific symptoms, root cause indicators, and side effects to check). The prompt correlates these signals to answer three questions: Did the symptoms stop? Is the root cause indicator neutralized? Are there unexpected side effects? Do not use this prompt during active triage, before a fix is deployed, or as a diagnostic tool for finding the root cause. It is a verification gate, not an investigation tool.

The output is a structured verification report with a clear status—verified, inconclusive, or failed—and supporting evidence for each signal. In high-severity incidents, this report should feed into a human approval step before incident closure. The prompt is designed to be wired into an automated harness that validates the output schema, retries on malformed responses, and logs the verification decision for the incident record. If the prompt returns 'inconclusive' due to missing data, the next step is to gather the required signals and re-run verification, not to guess.

This template is the core verification prompt you'll copy into your incident response automation. It is designed to receive structured incident data, remediation details, and post-action monitoring signals, then produce a rigorous verification report. The report confirms whether symptoms ceased, root cause indicators resolved, and no new side effects emerged. Use this template as the starting point for your verification harness, replacing each placeholder with real data from your incident management and observability systems.

textCopy
You are an incident remediation verification analyst. Your task is to confirm whether a remediation action resolved an incident. You will receive incident details, the action taken, and post-action monitoring data. Produce a structured verification report.

## INCIDENT CONTEXT
[INCIDENT_DETAILS]

## REMEDIATION ACTION TAKEN
[REMEDIATION_ACTION]

## POST-ACTION MONITORING DATA
[MONITORING_DATA]

## VERIFICATION CRITERIA
[VERIFICATION_CRITERIA]

## OUTPUT SCHEMA
Produce a JSON object with the following structure:
{
  "verification_status": "PASSED" | "FAILED" | "INCONCLUSIVE",
  "symptom_cessation": {
    "status": "CONFIRMED" | "NOT_CONFIRMED" | "PARTIAL",
    "evidence": ["string"],
    "unresolved_symptoms": ["string"]
  },
  "root_cause_indicators": {
    "status": "RESOLVED" | "NOT_RESOLVED" | "UNCERTAIN",
    "evidence": ["string"],
    "lingering_indicators": ["string"]
  },
  "side_effect_analysis": {
    "new_anomalies_detected": boolean,
    "anomalies": ["string"],
    "impacted_services": ["string"]
  },
  "confidence_score": number (0.0 to 1.0),
  "recommendation": "CLOSE_INCIDENT" | "CONTINUE_MONITORING" | "ESCALATE" | "ROLLBACK",
  "evidence_summary": "string",
  "requires_human_review": boolean
}

## CONSTRAINTS
- Base all conclusions strictly on the provided monitoring data. Do not speculate beyond the evidence.
- If monitoring data is insufficient to confirm resolution, set verification_status to "INCONCLUSIVE" and requires_human_review to true.
- Flag any new anomalies even if they appear unrelated to the original incident.
- Set confidence_score below 0.7 if any criterion cannot be fully confirmed.
- If recommendation is "ROLLBACK" or "ESCALATE", requires_human_review must be true.

## RISK LEVEL
[RISK_LEVEL]

## EXAMPLES
[FEW_SHOT_EXAMPLES]

To adapt this template, start by mapping your observability data sources into the [MONITORING_DATA] placeholder. This should include pre- and post-action metric snapshots, log patterns, alert status, and any synthetic check results. Define your [VERIFICATION_CRITERIA] as explicit, measurable conditions: for example, 'error rate below 0.1% for 5 minutes' or 'all health check endpoints returning 200.' The [RISK_LEVEL] field should be set to LOW, MEDIUM, HIGH, or CRITICAL based on your incident severity framework, and it should influence the confidence threshold required for auto-closure. The [FEW_SHOT_EXAMPLES] placeholder is critical for output consistency—provide at least two examples showing a PASSED scenario with clear evidence and a FAILED scenario with side effects. After copying this template, pair it with the implementation harness described in the next section to add validation, retries, and human review gates before deploying to production.

This prompt is designed to be the final gate in an automated or semi-automated incident remediation pipeline. It should be invoked only after a remediation action—such as a configuration change, a deployment rollback, a database failover, or an infrastructure patch—has been executed and the system has had enough time to stabilize. The harness must provide the prompt with structured, multi-signal evidence: the original incident description, the remediation steps taken, and a time-windowed snapshot of current system telemetry (metrics, logs, alerts, and error budgets). The prompt's job is not to execute a fix but to produce a defensible verification report that a human incident commander can use to decide whether to declare the incident resolved or to continue investigating.

To integrate this into a production workflow, build a verification pipeline that executes in three stages. First, a data collection stage gathers the required inputs: the incident ticket from your ITSM (e.g., PagerDuty, ServiceNow), the remediation runbook steps from your automation log (e.g., Ansible, Terraform, or a CI/CD pipeline), and a time-boxed telemetry snapshot from your observability stack (e.g., Datadog, Prometheus, Grafana). Second, an assembly stage populates the prompt's placeholders—[INCIDENT_DESCRIPTION], [REMEDIATION_STEPS], [TELEMETRY_SNAPSHOT], and [VERIFICATION_CRITERIA]—and sends the request to a capable reasoning model (GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro). Third, a validation stage parses the JSON output against a strict schema that requires a verdict field (one of RESOLVED, UNRESOLVED, or INCONCLUSIVE), a list of verified_symptoms, a list of unresolved_anomalies, and a confidence_score. If the output fails schema validation, retry once with the validation error appended to the prompt as a correction hint. If it fails twice, escalate to a human with the raw output and the validation errors.

For high-severity incidents (SEV1/SEV2), the harness must enforce a human-in-the-loop gate regardless of the model's verdict. Even if the prompt returns RESOLVED with high confidence, route the verification report to the incident commander for explicit approval before closing the incident. Log the full prompt, the model's response, the validator result, and the human decision to your audit trail. Avoid wiring this prompt directly to an auto-close action without human review—false negatives (declaring an incident resolved when it is not) can extend outages and erode trust in the automation. For lower-severity incidents, you can configure the harness to auto-close on a RESOLVED verdict with a confidence score above a defined threshold (e.g., 0.9), but always log the evidence for post-hoc review.