PII Redaction Confidence Scoring and Review Prompt Template

This prompt is for quality assurance (QA) teams, trust and safety engineers, and compliance reviewers who need to build a human review loop on top of automated PII redaction. The core job-to-be-done is not just redacting PII, but scoring the model's confidence in each redaction decision and flagging borderline cases where the model might have missed sensitive data or over-redacted benign text. You should use this prompt when your system already has a first-pass PII detection mechanism in place and you need a structured, auditable way to route ambiguous outputs to a human queue. The ideal reader is someone responsible for the safety and recall of a production AI pipeline, not an end-user casually sanitizing a single document.

Do not use this prompt as your primary PII detector. It is designed to operate on outputs that have already been flagged or redacted by a dedicated detection step. Its value is in adding a confidence layer and generating a review-ready summary, not in performing the initial scan. You should also avoid this prompt if you cannot tolerate any latency from a human review step; the output is designed to be consumed by a review tool or queue, not to be the final word in a real-time streaming application. The prompt assumes you have a [REDACTED_TEXT] input and a [DETECTION_METADATA] payload that includes the original spans, detected types, and the initial redaction actions taken.

Before wiring this into production, ensure you have a clear definition of what constitutes a 'borderline' case for your organization. This prompt uses configurable [CONFIDENCE_THRESHOLDS] and [RISK_LEVEL] parameters to decide what gets escalated. If your risk tolerance is zero for a specific PII category like credit card numbers, you can set the threshold to escalate everything in that category regardless of the model's score. The next step after reading this section is to examine the prompt template and adapt the output schema to match your review tool's API contract.

This template is designed to be dropped into a post-redaction review pipeline. It takes a redacted output, the original text, and a list of redaction decisions, then scores each decision's confidence, flags borderline cases, and produces a structured summary suitable for a human review queue. The prompt assumes you have already run a PII detection and redaction step—this is the quality assurance layer that decides which redactions need a second look.

textCopy
You are a PII redaction quality auditor. Your job is to review redaction decisions, assign confidence scores, and flag cases that need human review.

## INPUT

Original Text:

[ORIGINAL_TEXT]

codeCopy

Redacted Text:

[REDACTED_TEXT]

codeCopy

Redaction Decisions (JSON array of objects with `original`, `redacted`, `category`, `span_start`, `span_end`):
```json
[REDACTION_DECISIONS]

TASK

For each redaction decision, evaluate whether the redaction was correct and assign a confidence score from 0.0 to 1.0.

Use these scoring guidelines:

• 1.0: Definitely PII. Clear, unambiguous match (e.g., a well-formed email address, a 9-digit SSN in expected format).
• 0.8–0.9: Likely PII. Strong contextual evidence but minor ambiguity (e.g., a name that could be a company name).
• 0.5–0.7: Uncertain. Could be PII or benign text. Needs human review.
• 0.0–0.4: Likely not PII. Probable false positive (e.g., a number that matches SSN format but is clearly a product code in context).

For each decision, also classify it as:

• true_positive: Correctly identified PII
• false_positive: Incorrectly flagged benign text as PII
• false_negative_note: If you see PII in the original that was NOT in the redaction decisions list, add it as a separate finding

CONSTRAINTS

• Do not invent redaction decisions that aren't in the input list, except for false negative findings.
• For false negative findings, include the exact span and category of missed PII.
• If the redacted text does not match the original text minus redacted spans, flag it as a redaction_artifact error.
• Pay special attention to [HIGH_RISK_CATEGORIES]—these require human review if confidence is below 0.9.

OUTPUT SCHEMA

Return a JSON object with this exact structure:

jsonCopy
{
  "review_id": "string",
  "summary": {
    "total_decisions": 0,
    "true_positives": 0,
    "false_positives": 0,
    "false_negatives_found": 0,
    "redaction_artifacts": 0,
    "requires_human_review": true,
    "human_review_reason": "string explaining why review is needed, or null"
  },
  "decisions": [
    {
      "index": 0,
      "original": "string",
      "redacted": "string",
      "category": "string",
      "confidence": 0.0,
      "classification": "true_positive | false_positive",
      "rationale": "string",
      "needs_review": true
    }
  ],
  "false_negatives": [
    {
      "original_span": "string",
      "category": "string",
      "span_start": 0,
      "span_end": 0,
      "rationale": "string"
    }
  ],
  "redaction_artifacts": [
    {
      "description": "string",
      "location_in_redacted": "string"
    }
  ]
}

EXAMPLES

[EXAMPLES]

To adapt this template, replace the square-bracket placeholders with your pipeline's actual values. [ORIGINAL_TEXT] and [REDACTED_TEXT] are the before-and-after strings from your redaction step. [REDACTION_DECISIONS] is the JSON array your detector produced—each object must include the original span, the replacement, the PII category, and character offsets. [HIGH_RISK_CATEGORIES] should be a comma-separated list of categories that always trigger human review below 0.9 confidence, such as SSN, CREDIT_CARD, CREDENTIAL. [EXAMPLES] should contain 2–3 few-shot examples showing correct scoring for ambiguous cases—this is critical for calibrating the model's confidence thresholds to your organization's risk tolerance.

Before deploying this prompt, validate that your redaction decisions input matches the expected schema exactly. A common failure mode is mismatched span offsets between the original text and the redaction decisions array, which causes the model to flag spurious redaction_artifact errors. Test with a golden dataset of 20–30 known PII cases that includes both clear-cut and ambiguous examples, and measure inter-rater agreement between the model's confidence scores and your human reviewers. If agreement on borderline cases drops below 80%, adjust the scoring guidelines or add more few-shot examples. For high-risk domains like healthcare or finance, always route any decision with confidence below 0.9 to a human review queue—do not auto-approve borderline redactions.

This prompt is not a standalone detector—it is a scoring and review triage layer that sits behind an initial PII detection pass. The implementation harness should treat the prompt as a decision-support tool that assigns confidence scores, flags borderline cases, and produces structured review summaries for a human-in-the-loop queue. Wire it into your application after a detection step has already identified potential PII spans, and before any redacted output reaches logs, databases, or user-facing surfaces. The prompt expects pre-annotated text with candidate PII spans and their proposed redaction decisions; it does not perform the initial scan itself.

Integration flow: (1) Run your primary PII detector (regex, NER model, or a separate detection prompt) to identify candidate spans. (2) Package each candidate span with its surrounding context window, the proposed redaction action, and the detector's raw confidence into the [CANDIDATE_SPANS] input. (3) Call this scoring prompt with a structured output schema that enforces confidence_score (0.0–1.0), review_required (boolean), review_reason (string), and recommended_action (enum: redact, keep, escalate). (4) Validate the response against the schema before processing—reject malformed JSON and retry once with the error message appended to [CONSTRAINTS]. (5) Route all spans where review_required: true or confidence_score < [THRESHOLD] to a human review queue. (6) Log every decision with the model's confidence, the reviewer's final action, and a timestamp for audit trail and false positive/false negative tracking.

Model choice and latency: Use a fast, instruction-following model (GPT-4o-mini, Claude Haiku, or equivalent) for the scoring pass—this is a classification and justification task, not a generation task. If latency is critical for streaming outputs, batch candidate spans and score them asynchronously before the redacted text is committed. For high-compliance domains (healthcare, finance), require a second reviewer or a senior model pass on all escalated cases. Avoid: skipping schema validation on the model response, routing decisions without logging the confidence score, and treating the model's judgment as final without human review for borderline cases. The next step is to build your review queue UI and wire the structured output fields directly into the reviewer's decision form.