PII Leak Detection in Assembled Prompt

This prompt acts as a secondary, model-graded security scanner for your prompt assembly pipeline. Its job is to inspect the fully assembled prompt string—including system instructions, retrieved context, tool schemas, and user input—to detect Personally Identifiable Information (PII), secrets, and other sensitive data before transmission to a model provider. Use this when you need a programmatic gate that catches patterns deterministic regex-based redaction at the application layer might miss, such as PII embedded in retrieved documents, malformed API keys that slipped through a linter, or email addresses inside code snippets that a static rule didn't flag. The ideal user is a security or privacy engineer integrating this check into a CI/CD pipeline, a pre-inference request hook, or an observability platform that logs prompt assembly traces.

This is not a replacement for deterministic redaction. You must still strip known PII patterns at the application layer before assembly. This prompt is the safety net that catches what your regex missed. It is also not a general-purpose data classifier, a compliance audit tool, or a substitute for data loss prevention (DLP) infrastructure. Do not use it as the sole defense for regulated data under GDPR, HIPAA, or PCI-DSS. The model's detection is probabilistic and can miss obfuscated or novel PII formats. Always pair this with deterministic scanning, and route high-severity findings to a human review queue before blocking or redacting automatically. For production systems, log every scan result with the prompt's trace ID, the model's verdict, and the specific spans flagged so you can audit false negatives and refine your regex rules over time.

Before wiring this into your pipeline, define your severity thresholds and action map. A low-confidence phone number match in a public FAQ document might warrant a log entry; a high-confidence API key match in a customer support transcript should halt the request and trigger an alert. Start by running this prompt against a golden dataset of prompts containing known PII, benign structured data that looks like PII (e.g., fake credit card numbers in documentation), and edge cases like PII split across multiple context blocks. Measure false positive and false negative rates against your deterministic scanner. Use those results to decide whether this prompt gates requests, enriches logs, or feeds a human review queue. The next section provides the copy-ready template you can adapt and embed directly into your assembly harness.

This template instructs the model to act as a security auditor, scanning the fully assembled prompt for sensitive data patterns before transmission to the model. It covers email addresses, phone numbers, credit card numbers, API keys, social security numbers, and other regulated identifiers. The prompt is designed to be dropped into a preflight validation pipeline where it inspects the final prompt string that would otherwise be sent to a generative model.

textCopy
You are a security auditor specialized in detecting sensitive data in text before it reaches external systems. Your task is to scan the provided assembled prompt for personally identifiable information (PII), secrets, and regulated data patterns.

## INPUT
Analyze the following assembled prompt text:

[ASSEMBLED_PROMPT]

## DETECTION RULES
Scan for these categories with the specified patterns:

1. **EMAIL**: Patterns matching username@domain.tld
2. **PHONE**: US and international phone number formats including (XXX) XXX-XXXX, XXX-XXX-XXXX, +X-XXX-XXX-XXXX
3. **CREDIT_CARD**: 13-19 digit sequences that pass Luhn validation patterns, with or without separators
4. **SSN**: XXX-XX-XXXX pattern (US Social Security)
5. **API_KEY**: Patterns matching common API key formats (sk-, pk-, api-, key-, token prefixes, long base64 strings, hex strings > 32 chars)
6. **IP_ADDRESS**: IPv4 and IPv6 addresses
7. **DOB**: Date patterns in MM/DD/YYYY, DD-MM-YYYY, or written forms near age indicators
8. **ADDRESS**: Street address patterns with numbers, street names, city, state, ZIP
9. **PASSPORT**: Alphanumeric patterns matching passport number formats
10. **DRIVERS_LICENSE**: State-specific license number patterns

## CONSTRAINTS
- Do NOT flag placeholder text like [USER_NAME], {{variable}}, or template tokens
- Do NOT flag example/test data that is clearly synthetic (e.g., test@example.com, 555-0100)
- Do NOT flag benign structured data like UUIDs, database IDs, or non-secret hex strings
- If uncertain about a match, mark it as LOW severity with a note
- Consider context: a 9-digit number in an error log is not necessarily an SSN

## OUTPUT_SCHEMA
Return ONLY valid JSON with this structure:

{
  "scan_result": {
    "findings": [
      {
        "category": "EMAIL|PHONE|CREDIT_CARD|SSN|API_KEY|IP_ADDRESS|DOB|ADDRESS|PASSPORT|DRIVERS_LICENSE",
        "severity": "CRITICAL|HIGH|MEDIUM|LOW",
        "match": "the exact matched text",
        "position": {"start": 0, "end": 0},
        "context_snippet": "surrounding text for verification",
        "recommended_action": "REDACT|MASK|REMOVE|REVIEW",
        "confidence": 0.0-1.0
      }
    ],
    "summary": {
      "total_findings": 0,
      "critical_count": 0,
      "high_count": 0,
      "medium_count": 0,
      "low_count": 0,
      "requires_human_review": true|false
    },
    "redaction_map": [
      {
        "original": "matched text",
        "replacement": "[REDACTED_EMAIL]|[REDACTED_PHONE]|etc",
        "category": "EMAIL|PHONE|etc"
      }
    ]
  }
}

## INSTRUCTIONS
1. Scan the entire [ASSEMBLED_PROMPT] systematically
2. For each finding, determine severity: CRITICAL for credentials/secrets, HIGH for direct PII, MEDIUM for potentially sensitive, LOW for ambiguous matches
3. Set requires_human_review to true if any CRITICAL or HIGH findings exist
4. Provide a redaction_map with safe replacements for every finding
5. If no findings exist, return the JSON with empty findings array and zero counts

To adapt this template, replace [ASSEMBLED_PROMPT] with the actual prompt string your system constructs before inference. The detection rules can be extended or narrowed based on your jurisdiction's data protection requirements. For production use, consider adding [EXCLUDED_PATTERNS] as an additional input to pass known-safe strings that should not trigger false positives, such as internal test identifiers or synthetic data markers used in your development environment. The confidence scoring enables downstream systems to apply different thresholds based on risk tolerance.

The PII leak detection prompt is designed to run as a preflight guard in your prompt assembly pipeline—after all variables, context, and evidence are substituted into the final prompt string but before that string is transmitted to the inference model. This placement ensures that no sensitive data reaches the model provider's infrastructure. The harness should intercept the assembled prompt, invoke the detection prompt, parse the structured output, and either block the request, redact matched spans, or escalate for human review depending on severity classification and your organization's data handling policy.

For a production implementation, wrap the prompt call in a validation layer that enforces the output schema. The detection prompt returns a JSON object with a redaction_map array and a severity_classification field. Your harness must validate that every entry in the redaction map contains a span_start, span_end, match_type, and confidence field. If the model returns malformed JSON or missing required fields, retry once with a repair prompt that includes the raw output and schema constraints. Log every detection event—including the match type, severity, and whether the request was blocked, redacted, or allowed—to your observability platform with the prompt's trace ID attached. This audit trail is essential for security reviews and for tuning false positive thresholds over time.

Model choice matters here. Use a fast, cheaper model for this preflight check (such as GPT-4o-mini, Claude Haiku, or a fine-tuned small classifier) because the task is narrow and latency directly adds to your end-to-end request time. Avoid routing this check to the same large model that will handle the primary task, as that defeats the purpose of keeping PII out of the primary model's context. If your system processes high volumes, consider batching detection requests or implementing a regex pre-filter for high-precision patterns (credit card numbers, API key formats) before invoking the LLM-based detector. The LLM detector should handle ambiguous cases—like email-like strings in code comments or phone-number-like patterns in identifiers—that regex alone would misclassify.

For high-risk domains (healthcare, finance, legal), implement a human review queue for any detection with severity_classification: high and confidence below 0.90. The harness should serialize the flagged prompt segment, the detection output, and surrounding context into a review ticket. Do not send the full prompt to the reviewer if it contains the suspected PII—show only the matched span and enough context to make a judgment. After review, feed the decision (block, redact, allow) back into your detection eval dataset to improve future threshold tuning. For low-risk, high-confidence detections, automated redaction with a placeholder token like [REDACTED_EMAIL] or [REDACTED_CCN] is usually sufficient, but always verify that redaction didn't break the semantic coherence of the prompt before proceeding to inference.

Finally, test this harness with a dedicated eval suite before deployment. Your eval dataset should include prompts with known PII injected at various positions, benign prompts with PII-like patterns (e.g., user@example in documentation, 1234-5678-9012-3456 as a transaction ID), and edge cases with overlapping or malformed sensitive data. Measure false positive rate on benign structured data and false negative rate on obfuscated PII (e.g., john dot doe at gmail dot com). Run these evals as part of your CI/CD pipeline whenever the detection prompt or assembly logic changes. A silent PII leak into model inference is a data breach—treat this harness with the same rigor you'd apply to any security-critical code path.