Error Message Data Leakage Audit Prompt

This prompt is designed for SREs and platform engineers who need to audit production error messages and stack traces for accidental data exposure. Its primary job is to analyze a raw log entry—specifically an error span isolated from a distributed trace—and determine whether the error text contains user data, credentials, personally identifiable information (PII), or internal system details that should have been sanitized before logging. The ideal user is an engineer conducting an incident postmortem, a privacy reviewer performing a routine trace audit, or a developer building an automated trace-scanning pipeline that flags potential leakage events for human review. The prompt distinguishes between diagnostic data that is necessary for debugging (such as a null pointer exception on an internal class name) and user data that represents a leakage event (such as a full user object serialized into a stack trace).

Use this prompt when you already have isolated error spans from a trace aggregator like Datadog, Honeycomb, or OpenTelemetry, and you need a structured finding with a severity classification and a remediation recommendation. It is not a general-purpose log parser, a static secret scanning tool (like truffleHog or GitGuardian), or a replacement for automated PII redaction in your logging pipeline. It works best on structured error objects with a defined error.message, error.stack_trace, and error.type field, rather than on unstructured multi-line log blobs. The prompt expects you to provide the error span as the [INPUT] and any relevant data handling policies as the [CONTEXT]. It will not scan for secrets in source code repositories, nor will it detect leakage in non-error log lines like info-level request payloads.

Do not use this prompt for real-time blocking of error logs in a hot path; the latency of an LLM call is unsuitable for inline log emission decisions. Instead, use it in asynchronous audit workflows, batch trace review jobs, or incident review checklists. For high-risk regulated environments, always route findings with a severity of high or critical to a human reviewer before taking automated action like rotating credentials or triggering a data breach notification. The prompt's output is a structured JSON finding, not a legal determination. Your next step after receiving a finding should be to validate the exposed data against your data catalog, determine if the exposure constitutes a reportable incident under your regulatory framework, and implement a fix in the logging code or error serialization layer to prevent recurrence.

This prompt is designed to be pasted directly into your audit workflow. It instructs the model to act as a privacy-focused SRE, reviewing a provided error log for any accidental exposure of user data, secrets, or internal system details. The goal is to distinguish between necessary diagnostic information and a data leakage incident that requires immediate remediation. Before using this prompt, ensure you have isolated the specific error message or stack trace you want to audit and have defined your organization's data classification and handling policies.

textCopy
You are a senior SRE and privacy engineer auditing production error logs. Your task is to analyze the provided error content for accidental data exposure. You must distinguish between safe diagnostic data (e.g., null pointers, type errors, connection timeouts) and sensitive data leakage (e.g., PII, API keys, internal file paths, database queries with parameters).

**Input Error Content:**
[ERROR_LOG_CONTENT]

**Organization Data Handling Policy Context:**
[DATA_CLASSIFICATION_POLICY]

**Output Requirements:**
Respond with a valid JSON object conforming to this schema:
{
  "error_type": "string (e.g., Stack Trace, Database Error, API Response)",
  "exposed_data_instances": [
    {
      "data_fragment": "string (the exact text that is a concern)",
      "data_category": "string (e.g., PII, Secret, Internal Path, Query Parameter)",
      "severity": "string (Critical, High, Medium, Low)",
      "should_be_sanitized": boolean,
      "sanitization_suggestion": "string (if true, describe how to mask or remove it)"
    }
  ],
  "is_diagnostic_only": boolean,
  "overall_assessment": "string (summary of findings and recommended action)"
}

**Constraints:**
- If no sensitive data is found, return an empty array for `exposed_data_instances` and set `is_diagnostic_only` to true.
- Do not hallucinate data categories. If a fragment is ambiguous, flag it as `Potential` and set the severity to `Low`.
- Pay special attention to stack traces that include function arguments, database connection strings, or raw HTTP request/response bodies.

To adapt this prompt, replace the [ERROR_LOG_CONTENT] placeholder with the raw text from your log aggregator or exception tracker. The [DATA_CLASSIFICATION_POLICY] placeholder should be filled with a concise summary of what your organization considers sensitive (e.g., 'Any email address, user ID, or text after "password=" is critical'). For high-risk production environments, always route Critical severity findings to a human review queue before automated redaction is applied to prevent data loss from false positives. A common failure mode is the model missing PII embedded in long, URL-encoded strings; consider pre-processing the log to decode URLs before passing them to this prompt.

This prompt is designed to operate as a batch audit step within a log ingestion or SIEM pipeline, not as a real-time interceptor. Wire it to run against sampled error logs pulled from your observability platform (e.g., Splunk, Datadog, Elasticsearch) on a scheduled cadence—hourly for high-traffic services, daily for lower-risk systems. The prompt expects a single error message or stack trace as [INPUT] and returns a structured JSON finding. Your harness must handle log batching, deduplication (by error fingerprint), and rate limiting to avoid overwhelming the model API. For high-cardinality error streams, pre-filter to unique error signatures before sending to the model to control cost and latency.

Validation and retry logic is critical because the output schema drives downstream alerting. Implement a strict JSON schema validator that checks for the required fields: error_type, exposed_data_category, exposed_data_sample (hashed or truncated), should_have_been_sanitized (boolean), and severity. If the model returns malformed JSON or missing fields, retry once with a repair prompt that includes the raw output and the expected schema. If the retry also fails, log the raw response to a dead-letter queue for manual review. For severity values of high or critical, route the finding to a human review queue (e.g., Jira ticket, PagerDuty alert) before any automated redaction or deletion is triggered. Never auto-remediate based solely on the model's classification—false positives on exposed_data_category can suppress legitimate debugging information.

Model choice and context window considerations: Use a model with strong structured output capabilities (GPT-4o, Claude 3.5 Sonnet) and enable JSON mode or structured outputs if available. The prompt's [CONSTRAINTS] section should be populated with your organization's specific data classification policy—define what constitutes PII, secrets, internal IPs, and stack addresses in your context. For stack traces that exceed the model's context window, truncate from the middle (preserving the error type at the top and the root cause at the bottom) and add a [TRUNCATED] marker. Do not send raw stack traces to external model APIs unless you have a DPA in place and the trace has been pre-scrubbed of known secrets using regex-based redaction as a first pass. The model audit is a second-layer review, not the primary sanitization step.

Logging and audit trail: Every model call must be logged with a correlation ID that links back to the source log entry. Store the full prompt, model response, validator result, and reviewer decision in an immutable audit table. This is essential for compliance evidence if a data leakage incident is later discovered. For high-risk environments, run a weekly reconciliation query that compares the model's findings against a manual spot-check of 5% of audited errors to detect drift in classification accuracy. If the model starts missing credential exposures or over-flagging benign diagnostic data, recalibrate the [CONSTRAINTS] and [EXAMPLES] sections of the prompt before the next audit cycle.