PII Detection Prompt for Trace Logs

Privacy engineers and compliance officers use this prompt to audit individual production AI trace spans for accidental exposure of personally identifiable information (PII). A trace span might capture raw user input, a retrieved document chunk, tool-call arguments, or a generated output—any of which can contain names, emails, phone numbers, credentials, or financial data that should not persist in observability storage. The prompt accepts a single log entry or trace span and returns a structured finding that identifies the PII entity type, suggests a concrete redaction, and assigns a severity level based on the data category and exposure context. This transforms an unstructured, high-volume log stream into actionable, reviewable findings that can feed a batch audit pipeline or a real-time guardrail before traces are written to long-term storage.

The ideal user is a privacy engineer or compliance officer who already has access to production trace data and needs a repeatable, automatable detection layer. The prompt requires a specific trace span as input, along with optional context such as the data classification policy, a list of allowed data categories, and the expected output schema. It is designed to be wired into a larger harness: a script or middleware that fans out trace spans, calls the model with this prompt, validates the structured output, and routes high-severity findings for human review. The prompt works best when the input span is self-contained and includes enough surrounding text to disambiguate common nouns from actual PII—for example, distinguishing the name 'Grace' in a user message from the programming language concept.

Do not use this prompt as your only PII control. It is a detection layer, not a replacement for data classification at ingestion, encryption at rest, or access controls on your observability platform. The prompt can produce false negatives on obfuscated or indirect PII, and false positives on common nouns that resemble names or identifiers. Always pair it with a validation harness that checks the output schema, suppresses known false-positive patterns, and escalates ambiguous findings to a human reviewer. For high-risk regulated domains such as healthcare or finance, human review of all medium- and high-severity findings is mandatory before any automated redaction or deletion action is taken on the underlying trace data.

The following prompt template is designed to be pasted directly into your prompt layer. It instructs the model to act as a privacy audit assistant, scanning a provided trace log entry for personally identifiable information (PII). The template uses square-bracket placeholders for all dynamic inputs, such as the log content, the specific PII categories to scan for, and the desired output structure. Before integrating this into a production system, you must replace each placeholder with your actual data, schemas, and constraints. This template is the core instruction set; the surrounding application harness is responsible for validation, retries, and secure handling of the log data itself.

textCopy
You are a privacy audit assistant. Your task is to scan the provided [TRACE_LOG_ENTRY] for accidental exposure of personally identifiable information (PII).

Scan for the following PII categories: [PII_CATEGORIES].

For each potential PII finding, you must produce a structured output object that conforms to the following JSON schema:
[OUTPUT_SCHEMA]

Apply these constraints to your analysis:
[CONSTRAINTS]

Here are a few examples of correct input-output pairs:
[EXAMPLES]

If you are unsure about a finding, set its confidence score to 'LOW' and flag it for human review. Do not include any explanatory text outside the structured JSON output.

To adapt this template, start by defining the [TRACE_LOG_ENTRY] placeholder. This should be a single, complete log line or a serialized span object from your observability tool. The [PII_CATEGORIES] placeholder must be replaced with a concrete, comma-separated list of the entity types you are targeting, such as EMAIL, PHONE_NUMBER, CREDIT_CARD, or SSN. The [OUTPUT_SCHEMA] is critical for downstream processing; replace it with a strict JSON Schema definition that includes fields like entity_type, value_snippet, severity, and redaction_suggestion. The [CONSTRAINTS] placeholder should contain rules to reduce false positives, for example, 'Ignore common nouns and placeholder text like "John Doe" or "test@example.com".' Finally, provide 2-3 diverse [EXAMPLES] that demonstrate both a positive finding and a clean log entry to calibrate the model's behavior. After adapting the template, always test it with a golden dataset of known PII and non-PII log entries to validate its precision and recall before deployment.

This prompt is designed to be called programmatically as part of an automated trace auditing pipeline, not as a one-off manual review tool. The implementation harness should treat each trace span or log entry as a discrete unit of work, sending it to the model with the prompt template and collecting structured findings. Because the prompt operates on potentially sensitive data, the harness itself must run in a secured environment with access controls, audit logging, and data minimization—never log the raw prompt payloads that contain unmasked PII. The harness should batch trace entries by sensitivity tier when possible, processing public and internal entries first to reduce the blast radius of any misconfiguration before scanning confidential or restricted data.

Wire the prompt into a pipeline that follows this sequence: (1) extract candidate spans from your trace store using time-range and service filters, (2) apply a lightweight pre-filter to skip spans that are structurally incapable of containing PII—empty bodies, binary payloads, or spans with only machine-generated IDs—to reduce model call volume, (3) send each qualifying span through the prompt with a strict JSON output schema and response_format set to JSON mode where the model supports it, (4) validate the returned JSON against the expected schema with required fields: finding_id, entity_type, redaction_suggestion, severity, span_location, and confidence, (5) on validation failure, retry once with the error message appended to the prompt as a correction hint, and (6) route severity: critical and severity: high findings to a human review queue before any automated redaction is applied. For model choice, prefer a model with strong structured output discipline and low false-positive rates on entity detection—Claude 3.5 Sonnet or GPT-4o with strict JSON mode are suitable starting points. Avoid smaller or older models that conflate common nouns with PII, especially for name and location fields.

The eval harness should run alongside every production scan using a curated golden set of trace spans with known PII presence and absence. Include positive cases: plain-text emails, phone numbers, SSNs, credit card numbers, and API keys embedded in log messages. Include negative cases: common nouns that look like names ('Apple', 'Paris'), placeholder values ('test@example.com', '000-000-0000'), hex strings that are not secrets, and UUIDs. Measure precision and recall per entity type, and set a minimum recall threshold of 0.95 for critical entity types like credentials and financial account numbers before the pipeline can auto-escalate findings. Log every false positive and false negative to a review dashboard so the prompt template can be tuned over time. For obfuscated PII—base64-encoded secrets, split strings, or environment variable references—the eval set must include adversarial examples to measure detection gaps and inform whether a secondary decoding pre-processing step is needed before the prompt runs.