Log Pattern Anomaly Detection Prompt

This prompt is designed for platform engineers and SREs conducting offline investigation of distributed system health. The core job-to-be-done is comparing a structured, pre-processed batch of recent logs against a known, stable baseline of expected log patterns to surface anomalous error rates, new error signatures, and silent failures. The ideal user has already aggregated and sampled logs from their observability pipeline, defined a baseline representing normal operation, and needs a ranked anomaly report with confidence scores and suggested investigation paths rather than a simple threshold alert. This workflow assumes you are in a diagnostic or forensic context, not an active incident response with sub-second latency requirements.

To use this prompt effectively, you must provide two distinct, structured inputs: a baseline representing expected log patterns with their normal frequency and variance, and a recent batch of logs to compare against that baseline. The prompt expects these inputs to be pre-processed and sampled—raw, unbounded log streams will produce unreliable results and high token costs. The output is a ranked list of anomalies, each with a confidence score, a description of the deviation, and a suggested investigation path. This structure is valuable for weekly service reviews, post-deployment verification, and silent failure hunting where you need to prioritize engineering attention across dozens of services. The prompt does not replace your real-time alerting pipeline; it complements it by finding what your thresholds miss.

Do not use this prompt when the baseline is unknown, unrepresentable as a set of discrete patterns, or when you cannot tolerate the latency of an LLM inference call. It is also inappropriate for systems where log volume exceeds practical context windows without aggressive sampling, or where the cost of missing a single anomaly is catastrophic without human review. For real-time alerting, rely on stream processing and threshold-based systems. For environments where the baseline drifts rapidly, pair this prompt with a baseline regeneration workflow. Always validate the output against raw log evidence before escalating to incident response, and consider implementing a human review step for anomalies with confidence scores below a defined threshold.

This prompt template is designed to be wired into an automated monitoring pipeline or used manually by an on-call engineer during an investigation. It expects a structured baseline representing normal log behavior and a current window of log data to compare against. The model is instructed to act as a platform reliability engineer, focusing on statistical deviations, novel error signatures, and silent failures that would not trigger threshold-based alerts. The output is a ranked anomaly report with confidence scores and concrete investigation paths, not a generic summary.

codeCopy
You are a platform reliability engineer analyzing log streams for a distributed system. Your task is to compare the [CURRENT_LOG_WINDOW] against the [BASELINE_LOG_PATTERNS] and identify anomalous behavior.

## INPUT DATA

### Baseline Patterns (expected normal behavior)
[BASELINE_LOG_PATTERNS]

### Current Log Window (period under investigation)
[CURRENT_LOG_WINDOW]

### Context
- Service: [SERVICE_NAME]
- Environment: [ENVIRONMENT]
- Time Window: [TIME_RANGE]
- Recent Deployments: [RECENT_DEPLOYS]
- Known Incidents: [KNOWN_INCIDENTS]

## OUTPUT SCHEMA

Return a JSON object with the following structure:
{
  "summary": "One-sentence assessment of overall log health",
  "anomalies": [
    {
      "rank": 1,
      "type": "rate_spike | rate_drop | new_signature | silent_failure | pattern_shift",
      "pattern": "The specific log pattern or error signature",
      "baseline_frequency": "Expected occurrences per [TIME_UNIT]",
      "current_frequency": "Observed occurrences in current window",
      "deviation_factor": "Multiplier over baseline (e.g., 5.2x)",
      "confidence": 0.0-1.0,
      "first_seen": "ISO timestamp of first occurrence",
      "affected_components": ["component names"],
      "investigation_path": "Specific next step: which logs to check, which service to inspect, which metric to correlate",
      "related_code_paths": ["file paths or module names from repository context"]
    }
  ],
  "silent_failures": [
    {
      "signature": "Error pattern with no corresponding alert",
      "evidence": "Log lines demonstrating the gap",
      "suggested_alert_rule": "Proposed threshold or condition"
    }
  ],
  "correlation_hypotheses": [
    {
      "hypothesis": "Plausible causal chain linking anomalies to recent changes",
      "supporting_evidence": "Log lines, deploy timestamps, or config changes",
      "confidence": 0.0-1.0
    }
  ]
}

## CONSTRAINTS
- Only report anomalies where deviation_factor > [MIN_DEVIATION_THRESHOLD] or the pattern is entirely new.
- Do not flag expected fluctuations from [KNOWN_MAINTENANCE_WINDOWS] or [SCHEDULED_EVENTS].
- For each anomaly, provide at least one specific investigation path referencing actual log fields, service names, or code paths.
- If confidence is below [MIN_CONFIDENCE_THRESHOLD], mark the finding as "requires human review" and explain why.
- Do not invent log lines or patterns not present in the input data.
- If the current window contains no anomalies, return an empty anomalies array with a summary stating normal behavior.

## EXAMPLES

### Example Anomaly Output
{
  "rank": 1,
  "type": "new_signature",
  "pattern": "ConnectionResetError: [Errno 104] Connection reset by peer on /api/payments/process",
  "baseline_frequency": "0 per hour",
  "current_frequency": "47 occurrences in 15 minutes",
  "deviation_factor": "new",
  "confidence": 0.94,
  "first_seen": "2025-03-15T14:32:01Z",
  "affected_components": ["payment-gateway", "api-gateway"],
  "investigation_path": "Check payment-gateway upstream dependency health at 14:30 UTC. Correlate with deploy v2.4.1 to api-gateway at 14:28 UTC. Review connection pool config in api-gateway/config/upstreams.yaml.",
  "related_code_paths": ["api-gateway/config/upstreams.yaml", "payment-gateway/src/handlers/process_payment.py"]
}

To adapt this prompt, replace each square-bracket placeholder with your actual data. The [BASELINE_LOG_PATTERNS] should be a representative sample of normal log volume, error rates, and signature distribution from a comparable time window. The [CURRENT_LOG_WINDOW] is the period under investigation. Set [MIN_DEVIATION_THRESHOLD] based on your system's noise floor—typically 2x to 5x for high-traffic services. Set [MIN_CONFIDENCE_THRESHOLD] to control how aggressively the model surfaces uncertain findings; 0.7 is a reasonable starting point. Before deploying this into an automated pipeline, validate the output against a known-good baseline and a known-anomalous window to calibrate thresholds. For production use, always log the raw model output alongside the parsed JSON for auditability, and implement a schema validator that rejects malformed responses before they reach downstream alerting systems.

The log anomaly detection prompt is designed to operate as a stateless analysis step inside a larger observability pipeline. It expects a structured comparison between a current log window and a known baseline, so the application layer is responsible for fetching both datasets before calling the model. A typical harness runs on a cron or after a monitoring alert fires: it queries your log aggregator (e.g., Elasticsearch, Loki, Splunk) for the current time window, retrieves the corresponding baseline from a stored snapshot or historical query, formats both into the [CURRENT_LOGS] and [BASELINE_LOGS] placeholders, and then invokes the model. The prompt is not responsible for time-window selection, log filtering, or PII redaction—those must happen upstream.

Model choice and tool use. This prompt works best with models that have strong reasoning and structured output capabilities (GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro). Set response_format to json_schema (OpenAI) or use a structured output mode (Anthropic) that enforces the [OUTPUT_SCHEMA] you provide. The prompt does not require tool calls or RAG—it is a pure analysis task over provided data. If your log volumes exceed the model's context window, pre-summarize each log stream using log-pattern extraction (e.g., Drain, LogCluster) and pass the pattern summaries instead of raw lines. For high-cardinality services, consider sharding by service name and running the prompt in parallel per service, then merging anomaly reports.

Validation and retry logic. After the model responds, validate the output against the expected schema before surfacing it to an on-call engineer. Check that anomalies is an array, each entry has a non-empty signature and confidence_score between 0 and 1, and investigation_paths are present for high-confidence findings. If validation fails, retry once with a repair prompt that includes the validation error and the original output. If the retry also fails, log the raw response and escalate to a human with a VALIDATION_FAILURE flag. For high-severity environments, add a human-in-the-loop gate: any anomaly with confidence_score > 0.8 and severity == "critical" should create a ticket or page rather than auto-close.

Logging and observability. Wrap every model call with structured logging that captures: the time window queried, the number of log lines in current vs. baseline, the model used, token count, latency, validation status, and the anomaly count. This data feeds your prompt observability dashboard and lets you detect drift in model behavior over time. If you notice the model consistently flagging the same false-positive pattern, add it as a negative example in [EXAMPLES] or adjust the [CONSTRAINTS] to exclude known noisy signatures. Store prompt versions alongside your application code so you can correlate anomaly report changes with prompt updates during incident review.

What to avoid. Do not send raw, unredacted production logs to external model APIs without a PII and secret redaction layer. Do not use this prompt for real-time alerting with sub-second latency requirements—model inference adds 2–10 seconds. Do not treat the model's output as a definitive incident declaration; it is a prioritized investigation aid that still requires human judgment, especially for novel failure modes the baseline cannot capture. If your system requires deterministic, repeatable anomaly scoring, pair the prompt with a statistical method (e.g., z-score on error rate) and use the model only to explain and contextualize the statistical signal.