Log Completeness Assessment Prompt for Failure Scenarios

This prompt is designed for platform engineers and SREs who need to validate that their services produce sufficient diagnostic signals during controlled failure scenarios. The core job-to-be-done is turning raw log output from a chaos experiment or fault injection into a structured completeness report. You should use this prompt when you have already executed a failure injection—such as killing a pod, severing a network link, or exhausting a connection pool—and captured the resulting logs from all affected services. The prompt assumes you possess both the raw log output and a known expected schema or a defined set of required fields (e.g., correlation IDs, request context, upstream/downstream identifiers). It is ideal for pre-production release gates, where you need evidence that your observability stack will support rapid incident diagnosis before real users are impacted.

Do not use this prompt as a substitute for real-time log anomaly detection, live tailing tools, or streaming alert evaluation. It is a point-in-time audit, not a continuous monitoring system. The prompt also does not replace a full distributed trace analysis; it focuses on the textual log content rather than span topology or timing. You should avoid using it when the failure scenario is undefined or when you lack an expected schema—without a clear contract for what 'complete' means, the model will produce a generic critique rather than a precise gap analysis. For high-risk production environments, always pair the output of this prompt with a human review step, especially when the completeness report flags missing correlation IDs or truncated stack traces that could mask a latent incident response blind spot.

Before running this prompt, ensure your input logs are sanitized of any production PII or secrets. The prompt template includes a [CONSTRAINTS] placeholder where you can inject redaction rules, but the safest path is to scrub data before it reaches the model. Once you receive the structured completeness report, use it to update your logging standards, instrumentation libraries, or runbook expectations. If the report identifies repeated gaps, consider automating the check as a CI/CD gate using the schema validation harness described in the implementation section. The next step after reading this introduction is to review the prompt template, adapt the placeholders to your specific failure scenario and expected schema, and run it against a known-good and known-bad log sample to calibrate your expectations.

This prompt is the core of the Log Completeness Assessment workflow. It is designed to be pasted directly into your AI harness, with square-bracket placeholders replaced by real values at runtime. The template instructs the model to act as a platform engineering auditor, systematically checking a provided log sample against a defined schema of required diagnostic fields. The output is a structured JSON report that your application can parse, store, and use to trigger automated quality gates or generate tickets for instrumentation gaps.

textCopy
You are a platform engineering auditor. Your task is to analyze a log sample captured during a failure scenario and produce a structured completeness report.

## INPUT
Failure Scenario Description: [FAILURE_SCENARIO_DESCRIPTION]
Log Sample (plain text or JSON lines):

[LOG_SAMPLE]

codeCopy
Expected Service Topology (call path): [SERVICE_TOPOLOGY]

## OUTPUT SCHEMA
Return a single JSON object conforming to this structure:
{
  "assessment_id": "string",
  "overall_completeness_score": number (0-100),
  "critical_gaps": [
    {
      "gap_type": "missing_correlation_id" | "truncated_stack_trace" | "absent_request_context" | "missing_upstream_identifier" | "missing_downstream_identifier" | "missing_timestamp" | "missing_log_level" | "other",
      "severity": "critical" | "high" | "medium" | "low",
      "location": "string (line number or span ID)",
      "evidence": "string (excerpt from log showing the gap)",
      "recommendation": "string (actionable fix)"
    }
  ],
  "field_completeness": {
    "correlation_id_present": boolean,
    "request_context_present": boolean,
    "upstream_service_id_present": boolean,
    "downstream_service_id_present": boolean,
    "stack_trace_complete": boolean,
    "timestamp_iso8601": boolean,
    "log_level_present": boolean
  },
  "propagation_chain": [
    {
      "service_name": "string",
      "correlation_id_found": boolean,
      "span_id": "string or null"
    }
  ],
  "summary": "string (plain-language summary of findings)"
}

## CONSTRAINTS
- Only report a gap if you have direct evidence from the provided log sample. Do not speculate about missing data you cannot see.
- If a field is present but empty or null, mark it as absent.
- For the propagation chain, list every service in the expected topology and note whether a correlation ID was observed.
- The overall_completeness_score should reflect the proportion of required fields present and the severity of any critical gaps.
- If the log sample is unparseable or empty, return a valid JSON object with overall_completeness_score set to 0 and explain the parsing failure in the summary.

## RISK LEVEL
High. Incomplete logs during failure scenarios directly impact incident response time and diagnostic accuracy. Human review of this report is required before closing any associated incident or instrumentation task.

To adapt this template, replace the placeholders with real data from your test harness. [FAILURE_SCENARIO_DESCRIPTION] should be a brief, structured description of the injected failure, such as "Simulated network partition between service A and the inventory database." [LOG_SAMPLE] is the raw log output captured during that test, which can be a multiline string or a JSON-lines blob. [SERVICE_TOPOLOGY] must be a clear, ordered list of expected service hops for the request path under test, for example "api-gateway -> order-service -> payment-service -> notification-service." This topology is critical for the model to accurately populate the propagation_chain array and identify missing service identifiers. Before executing, ensure your harness validates that the model's output strictly conforms to the JSON schema; a failed parse should trigger a retry with a stronger format reminder.

The log completeness assessment prompt is designed to be invoked programmatically after a chaos engineering experiment, a fault injection test, or a canary deployment that triggers known failure modes. The typical integration point is a post-test hook in your CI/CD pipeline (e.g., a GitHub Actions step, a Jenkins post-build stage, or a Tekton task) that collects structured log output from your observability backend (Splunk, Elastic, Loki) for the time window of the injected failure. The prompt expects a JSON payload containing the raw log lines, the expected topology of the request path, and a schema of required fields per service. You should wrap the model call in a thin service that handles authentication, rate limiting, and response parsing.

Before sending the prompt, validate the input payload against a strict schema: ensure log_lines is a non-empty array of objects, expected_topology includes a valid service dependency graph, and required_fields maps each service to its mandatory log attributes (e.g., correlation_id, trace_id, upstream_service, downstream_service). On the output side, parse the model's JSON response and validate it against the expected completeness report schema. If the model returns malformed JSON, implement a retry loop with a repair prompt that includes the raw output and the schema definition. Log every model call—input payload, raw response, parsed report, and validation errors—to an audit table for traceability. For high-risk production audits, route reports with a completeness_score below a configurable threshold (e.g., 0.8) to a human reviewer via a Slack notification or a Jira ticket.

Model choice matters here. Use a model with strong JSON mode and long-context handling, such as Claude 3.5 Sonnet or GPT-4o, because log payloads can be large and the output schema is deeply nested. If you're processing high volumes, consider batching log windows and running assessments in parallel, but be mindful of rate limits and token costs. Avoid using this prompt as a real-time gate on critical path requests—it's an offline audit tool, not a live validation step. The next step after implementation is to build a dashboard that tracks completeness scores over time per service, so teams can see whether their logging discipline is improving or degrading with each release.