Log Level Appropriateness Audit Prompt

Platform engineers and SREs should use this prompt to perform a static audit of log severity levels across a codebase or a captured log stream. The primary job-to-be-done is identifying misclassified log statements—specifically, ERROR-level logs that represent normal operational conditions (and should be WARN or INFO) and DEBUG-level logs that surface critical state changes (and should be elevated to INFO or WARN). The prompt also cross-references each log statement against a provided team logging policy, producing a structured misclassification report that can be used to improve signal-to-noise ratio before an incident occurs.

This prompt is designed for pre-release observability checks, code review workflows, incident postmortem remediation, and CI pipeline quality gates. It assumes you are providing static log samples or source code extracts, not a live stream of telemetry. To use it effectively, you must supply the raw log lines or code snippets as [INPUT], the team's logging standards as [POLICY], and optionally a [CONTEXT] describing the system architecture. The output is a structured JSON report that flags each violation with a severity, a suggested correction, and a policy reference. Do not use this prompt for real-time log anomaly detection, security intrusion analysis, or PII redaction—those tasks require streaming, stateful, or specialized detection models that operate under different constraints.

Before integrating this prompt into a CI pipeline, ensure you have a validation harness that can parse the structured output and fail the build only on high-severity misclassifications. A common failure mode is the model flagging intentional ERROR-level logs used for alerting as false positives; mitigate this by including explicit alerting rules in your [POLICY] document. If your codebase generates a high volume of logs, batch the input and run the audit per service to stay within context window limits. For regulated environments, always require human review of the report before merging any log-level changes that could suppress audit-relevant events.

This prompt template is designed to be pasted directly into your AI harness or evaluation pipeline. It instructs the model to act as a log standards auditor, comparing provided log samples against your team's explicit severity-level policy. The output is a structured misclassification report, not a free-text opinion. Before using this prompt, ensure you have a clear, documented log-level policy (e.g., 'ERROR means a user-facing transaction failed and requires immediate human intervention'). Without a concrete policy, the model's judgment will be unreliable.

textCopy
You are a log standards auditor. Your task is to review a set of log statements and identify any that are misclassified according to the provided [LOGGING_POLICY].

For each log statement in the [LOG_SAMPLES], perform the following analysis:
1. Determine the actual severity based on the event described, ignoring the stated level.
2. Compare the actual severity against the stated level.
3. If they match, classify as 'correct'.
4. If they do not match, classify as 'misclassified' and specify the recommended level.

[LOGGING_POLICY]:
"""
[POLICY_DOCUMENT]
"""

[LOG_SAMPLES]:
"""
[LOG_ENTRIES]
"""

[OUTPUT_SCHEMA]:
Return a JSON object with a single key "audit_results" containing an array of objects. Each object must have the following fields:
- "log_line_index": (integer) the 0-based index of the log line from the input.
- "stated_level": (string) the original log level.
- "recommended_level": (string or null) the corrected log level if misclassified, otherwise null.
- "classification": (string) either "correct" or "misclassified".
- "rationale": (string) a brief, one-sentence explanation referencing the policy.

[CONSTRAINTS]:
- Do not alter the log message content.
- If a log statement's severity is ambiguous even after consulting the policy, classify it as 'correct' and note the ambiguity in the rationale.
- Only output the JSON object. Do not include any other text.

To adapt this template, replace the square-bracket placeholders with your specific context. The [POLICY_DOCUMENT] should be a verbatim copy of your team's internal standard, defining each level (e.g., DEBUG, INFO, WARN, ERROR, FATAL) with concrete, observable criteria. The [LOG_ENTRIES] should be a newline-separated list of raw log lines, including their stated level. For high-risk production systems, always route the output of this prompt to a human reviewer before accepting any automated severity changes. A common failure mode is the model over-flagging transient network errors as WARN instead of ERROR; mitigate this by explicitly defining retry and transient failure handling in your [POLICY_DOCUMENT].

This prompt is designed to be called programmatically, not used in a one-off chat interface. The primary integration point is a CI/CD pipeline that runs on pull requests touching logging statements, or a scheduled job that samples production logs and audits their severity levels against your team's written logging standards. The prompt expects a structured [LOG_STATEMENTS] input—typically a JSON array of objects, each containing the log line, the declared level, the source file, and surrounding code context. The output is a structured misclassification report that your harness can parse and surface as inline PR comments, Slack notifications, or dashboard widgets.

To wire this into an application, build a thin harness that: (1) extracts candidate log statements from changed files in a PR or from a log aggregator query (e.g., Elasticsearch, Loki, or CloudWatch), (2) formats them into the [LOG_STATEMENTS] JSON schema the prompt expects, (3) calls the model with the prompt template and a strict [OUTPUT_SCHEMA] specifying the expected JSON structure, (4) validates the response against that schema before acting on it, and (5) posts findings back to the PR or sends a summary to the team channel. For model choice, a fast instruction-following model like GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro works well. Avoid small or older models that may struggle with consistent JSON output under the policy compliance section. Set temperature=0 or very low to maximize deterministic severity judgments.

Validation and retry logic is critical here because a malformed audit report can block a pipeline or produce false positives. After receiving the model response, validate that every entry in the misclassifications array contains the required fields: log_statement, current_level, recommended_level, justification, and policy_rule_reference. If validation fails, retry once with the validation errors appended to the prompt as [CONSTRAINTS] feedback. If the second attempt also fails, log the raw response and escalate for human review rather than silently proceeding. For high-risk repositories (e.g., payment systems, healthcare data pipelines), always require a human to approve recommended level changes before they are applied—never auto-commit severity modifications. Log every audit run's model call, response, and validation outcome to your observability stack so you can track prompt drift and model behavior changes over time.