Incident Retrospective Observability Gap Analysis Prompt

This prompt is for SREs and platform engineers who need to analyze an incident and identify where observability failed. The job is to produce a structured gap analysis that maps the incident timeline to missing signals, slow detection, and diagnostic friction. Use this after an incident retrospective meeting when you have a timeline, chat logs, and access to monitoring dashboards. The output is a document that engineering teams can turn into a prioritized observability improvement backlog.

This prompt assumes you already have an incident timeline and are not starting from raw logs. It does not replace the human retrospective; it accelerates the observability-specific analysis that often gets skipped when teams focus on root cause and remediation actions. The prompt works best when you provide a detailed timeline with timestamps, actions taken, and the specific dashboards or tools queried during the incident. It is not designed for real-time incident response or for generating postmortem narratives. If you lack a structured timeline, use a separate timeline reconstruction prompt first.

Do not use this prompt when the incident is still active, when the retrospective has not yet produced a consensus timeline, or when the team lacks access to the monitoring tools referenced in the incident. The prompt will produce low-quality output if fed vague recollections instead of specific timestamps and tool names. For teams new to observability, pair this prompt with a human review step to validate that the identified gaps are actionable within your current tooling and budget constraints. The goal is a prioritized backlog, not an abstract wish list.

This is the core prompt for generating an Incident Retrospective Observability Gap Analysis. It is designed to be pasted directly into your LLM interface. The prompt forces the model to map a concrete incident timeline against your existing observability stack, identifying where signals were missing, detection was slow, or diagnosis was blocked by tooling or data gaps. The output is a structured gap analysis document, not a generic postmortem narrative.

textCopy
You are an SRE architect specializing in observability system design. Your task is to analyze a production incident and identify specific gaps in the observability stack that contributed to detection latency, diagnostic friction, or incomplete post-incident understanding.

## INCIDENT DATA
**Incident Summary:** [INCIDENT_SUMMARY]
**Timeline of Events (UTC):** [INCIDENT_TIMELINE]
**Services Affected:** [AFFECTED_SERVICES]
**Detection Method:** [DETECTION_METHOD]
**Total Time to Detect (TTD):** [TTD]
**Total Time to Resolve (TTR):** [TTR]

## CURRENT OBSERVABILITY STACK
**Logging:** [LOGGING_STACK_DESCRIPTION]
**Metrics:** [METRICS_STACK_DESCRIPTION]
**Tracing:** [TRACING_STACK_DESCRIPTION]
**Alerting:** [ALERTING_STACK_DESCRIPTION]
**Dashboards:** [KEY_DASHBOARDS_USED]

## CONSTRAINTS
- Do not propose solutions. Only identify gaps.
- Map every gap to a specific point on the incident timeline.
- Distinguish between missing signals, insufficient granularity, and tooling/UX friction.
- Classify each gap by its impact on Detection, Diagnosis, or Prevention.
- If the provided data is insufficient to determine a gap, state that explicitly instead of guessing.

## OUTPUT SCHEMA
Return a single JSON object with the following structure:
{
  "incident_id": "string",
  "analysis_summary": "A 2-3 sentence executive summary of the most critical observability failures.",
  "gaps": [
    {
      "gap_id": "string",
      "timeline_reference": "The specific timestamp or event from the timeline where this gap manifested.",
      "category": "missing_signal | insufficient_granularity | tooling_friction | alert_configuration | data_retention",
      "impact_phase": "detection | diagnosis | prevention",
      "description": "A clear, specific description of what was missing or insufficient.",
      "evidence_from_incident": "Quote or reference the specific timeline event that proves this gap.",
      "severity": "critical | high | medium | low"
    }
  ],
  "data_quality_notes": ["Any concerns about the completeness or accuracy of the provided incident data."]
}

Adapting the template: Replace every [PLACEHOLDER] with concrete data from your incident retrospective. The [INCIDENT_TIMELINE] is the most critical field—provide it in a structured, timestamped format. If your observability stack is complex, be specific about versions and configurations in the stack description fields. For high-severity incidents, run the output through a human review step before adding it to the official postmortem. The JSON output is designed to be ingested by a tracking system; validate it against the schema before closing the incident record.

This prompt is designed for a single-turn LLM call, not a multi-step agent. It works best when the incident data is pre-assembled before the call. The caller—typically an SRE tool, incident management bot, or retrospective automation—should gather the incident timeline, known signals, diagnostic steps taken, and any existing postmortem notes into a structured context block before invoking the model. The model's job is to produce a structured gap analysis, not to query live telemetry or interview responders.

Input assembly is the most critical pre-call step. Build the [INCIDENT_TIMELINE] as a chronologically ordered list of events with timestamps, severity changes, and responder actions. Populate [OBSERVABILITY_STACK] with the current tooling (e.g., Prometheus, Grafana, Datadog, OpenTelemetry), what each tool is configured to capture, and any known blind spots. Include [DIAGNOSTIC_FRICTION_LOG] as a list of moments where responders couldn't answer a question quickly—these are the raw material for gap identification. If any of these inputs are missing or incomplete, the output quality degrades sharply. Validate inputs before the call: check that the timeline has at least detection, diagnosis, and resolution phases; that the stack description includes what is not instrumented; and that friction log entries are specific (e.g., "could not determine which service was failing" rather than "hard to debug").

Output validation should run immediately after the model responds. Parse the JSON output and check: (1) every gap has a non-empty signal_missing, detection_delay_impact, and remediation field; (2) severity is one of the allowed enum values (critical, high, medium, low); (3) the timeline_to_gap_mapping array references timestamps that actually appear in the input timeline. If validation fails, retry once with the validation errors appended to the prompt as [PREVIOUS_VALIDATION_ERRORS]. If the second attempt also fails, flag the output for human review rather than retrying indefinitely. Log both the raw output and validation results for auditability.

Human review is required before any gap analysis is treated as final or used to prioritize engineering work. The model can miss domain-specific failure modes, misinterpret tool capabilities, or propose remediation that is infeasible given team constraints. Route the validated output to the incident owner or on-call SRE with a review checklist: confirm each gap is real, adjust severity if needed, and mark any gaps that are already known or accepted as tolerable risk. Only after human sign-off should the gaps be converted into tickets or roadmap items. Model choice: use a model with strong structured output capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid smaller models for this task—the reasoning required to connect timeline events to missing signals demands strong causal inference. Set temperature to 0 or very low (0.0–0.2) to maximize consistency across runs.

Do not feed live production telemetry directly into this prompt. The model has no real-time access and will hallucinate metrics. Instead, summarize the relevant signal gaps in the input block. Do not use this prompt during an active incident—it is a retrospective tool. During incidents, use runbook automation and human judgment. Save this analysis for the post-incident review window when the timeline is complete and responders are available to validate findings.