Data Masking Effectiveness Review Prompt Template

This prompt is designed for data platform and privacy engineering teams who need to validate that their production data masking rules are actually working. The core job-to-be-done is auditing masked trace data—such as log entries, API responses, or database query results captured in observability traces—to determine whether an adversary or a faulty downstream process could infer, reconstruct, or re-identify the original sensitive values. The ideal user is an engineer or compliance analyst who has access to both the raw trace spans and the masking policy configuration, and who needs a structured, repeatable way to assess residual risk before an auditor or a breach forces the issue.

You should use this prompt when you are reviewing a specific trace sample against a known masking rule (e.g., format-preserving encryption, tokenization, redaction, or pseudonymization). The prompt requires several concrete inputs to be effective: the original masking rule ID and its declared method, a sample of masked data from a production trace, and any available context such as adjacent fields, timestamps, or user-agent strings that could serve as auxiliary information for re-identification. The output is not a simple pass/fail; it produces a structured finding that includes a residual risk level (Low, Medium, High, Critical), a specific re-identification vector if one exists, and a recommendation for rule hardening. This makes the prompt suitable for integration into a CI/CD pipeline for masking policy changes or a periodic privacy review cadence.

Do not use this prompt as a substitute for formal cryptographic validation of encryption algorithms or for compliance certification. It is a heuristic review tool that relies on the model's ability to spot patterns, correlations, and inference paths in structured data, not a mathematical proof of anonymity. It is also not designed for real-time, high-throughput screening of every trace; it is best applied to a representative sample or traces flagged by a PII detection system. Before relying on the output, always pair it with a deterministic eval harness that tests the prompt against known re-identification scenarios—such as format-preserving masked emails where the domain is left in the clear, or tokenized names where the token length correlates with the original string length. If the prompt flags a Critical or High risk, the finding must be routed for human review and the masking rule should be quarantined until the vector is closed.

This prompt template is designed to be dropped into a review harness that feeds masked trace spans to a model for effectiveness analysis. It instructs the model to act as a privacy auditor, examining the provided masked data for format-preserving weaknesses, tokenization reversibility, and redaction gaps. The template uses square-bracket placeholders for all variable inputs, ensuring it can be adapted to different masking methods and data schemas without modification.

textCopy
You are a privacy audit assistant specializing in data masking effectiveness. Your task is to review a masked data sample from a production trace and determine whether the original sensitive value can be inferred, partially reconstructed, or re-identified.

## INPUT DATA
Masking Method: [MASKING_METHOD]
Original Data Type: [DATA_TYPE]
Masked Value: [MASKED_VALUE]
Surrounding Trace Context (if available): [TRACE_CONTEXT]

## CONSTRAINTS
- Do not attempt to brute-force or decode the value using external tools.
- Base your analysis solely on structural properties, format preservation, and contextual clues within the provided trace.
- If the masking method is tokenization, consider whether the token itself leaks information (e.g., consistent token length, prefix preservation).
- If the masking method is format-preserving encryption, analyze whether the output format constrains the possible input space.
- If the masking method is redaction, check for partial redaction, inconsistent redaction boundaries, or recoverable data in adjacent fields.

## OUTPUT SCHEMA
Return a valid JSON object with the following fields:
{
  "masking_rule_id": "string (identifier for the masking rule applied)",
  "residual_risk_level": "low | medium | high | critical",
  "re_identification_vector": "string (description of the specific weakness or inference path, or 'none identified')",
  "confidence": "low | medium | high",
  "evidence": ["string (specific observation from the masked value or context)"],
  "recommendation": "string (actionable fix for the masking rule)"
}

## ANALYSIS STEPS
1. Identify the masking method from the provided label and confirm it matches the observed masked value characteristics.
2. Examine the masked value for structural leakage: length preservation, character class preservation, prefix/suffix retention, format string patterns (e.g., email structure, date patterns).
3. Analyze the surrounding trace context for correlating fields that could narrow the possible original values (e.g., a masked name appearing alongside an unmasked zip code and age).
4. If tokenization is used, check whether the same token appears elsewhere in the trace with different context that could de-anonymize it.
5. Assess whether multiple masked fields in combination create a linkability or singling-out risk, even if each field individually appears safe.
6. Assign a residual risk level based on the worst-case re-identification scenario you can construct from the available data.

## EXAMPLES

Example 1: Format-Preserving Email Masking
Input: Masked Value: "j***@example.com", Data Type: email, Masking Method: partial-redaction
Output: {
  "masking_rule_id": "email-partial-redact-01",
  "residual_risk_level": "high",
  "re_identification_vector": "Local-part prefix 'j' combined with known domain and trace context containing first name 'James' allows reconstruction to 'james@example.com'.",
  "confidence": "high",
  "evidence": ["Single character prefix 'j' preserved", "Trace context contains unmasked first name field 'James' in same span", "Domain fully visible"],
  "recommendation": "Replace partial redaction with full local-part tokenization or hash-based pseudonymization. Do not preserve any prefix characters."
}

Example 2: Tokenized SSN
Input: Masked Value: "tok_abc123def456", Data Type: ssn, Masking Method: tokenization
Output: {
  "masking_rule_id": "ssn-tokenize-02",
  "residual_risk_level": "low",
  "re_identification_vector": "none identified",
  "confidence": "medium",
  "evidence": ["Token reveals no structural information about original SSN", "Token length does not correlate with SSN format", "No correlating fields in trace context"],
  "recommendation": "Current tokenization appears effective. Monitor for token reuse across traces that could enable cross-session correlation."
}

Now analyze the provided masked value and return only the JSON object.

To adapt this template, replace the placeholders with data extracted from your trace observability pipeline. The [MASKING_METHOD] should be populated from your data catalog or masking policy metadata. The [TRACE_CONTEXT] is critical for detecting linkability attacks—include adjacent span fields, session metadata, and any unmasked attributes that appear in the same trace window. For high-risk data types like SSNs, PHI, or financial account numbers, always route the model's output to a human reviewer before updating masking rules. Implement a validation step that confirms the output JSON conforms to the schema and that residual_risk_level is one of the four allowed enum values before accepting the result.

This prompt is designed to run as a batch review step inside a privacy trace pipeline, not as a real-time user-facing call. The typical integration point is a nightly or on-demand audit job that pulls masked trace spans from your observability store, runs them through the model, and writes structured findings to a compliance review queue. Because the prompt evaluates whether original values can be inferred or reconstructed from masked data, the harness must guarantee that the model never sees the original unmasked values—only the masked output and the masking rule metadata. This means your pipeline must strip or hash original values before the prompt is assembled, and the model call must occur in an environment with no access to the unmasked source of truth.

Validation and output enforcement are critical. The prompt asks for a JSON object with masking_rule_id, residual_risk_level (one of LOW, MEDIUM, HIGH, CRITICAL), and a re_identification_vector array. Your harness should validate the response against this schema immediately after generation. If the model returns an invalid risk level, a missing rule ID, or a malformed vector array, retry once with a repair prompt that includes the validation error. If the second attempt also fails, log the raw response and escalate to a human reviewer. For CRITICAL findings, the harness should automatically create a high-priority ticket and block the associated masking rule from production promotion until a human signs off. Do not auto-apply masking rule changes based on model output alone.

Model choice and configuration matter for this workflow. Use a model with strong reasoning capabilities and a low temperature (0.0–0.2) to maximize deterministic, repeatable assessments. The prompt includes a [MASKING_METHOD] field that accepts format_preserving, tokenization, or redaction—your harness should populate this from the masking rule's metadata so the model can apply method-specific inference checks. For format-preserving masks, the model should test whether the output preserves enough structure to reverse-engineer the original. For tokenization, it should check whether token collisions or lookup-table inference is possible. For redaction, it should assess whether surrounding context enables reconstruction. Log the model, temperature, prompt version, and trace span ID alongside every finding for auditability.

Eval and pre-release testing must cover all three masking methods with known-weak and known-strong examples. Build a golden dataset of 20–30 masked trace spans where you know the ground-truth re-identification risk. Include format-preserving masks that leak length and character class, tokenized values with predictable token sequences, redacted fields where adjacent context reveals the missing data, and strong masks that genuinely prevent inference. Run this eval suite before deploying any prompt change. A regression is defined as a finding that misses a known HIGH or CRITICAL risk, or that flags a known-safe mask as MEDIUM or above. Gate prompt releases on passing this eval suite with 100% recall on high-risk cases and at least 90% precision overall.

Human review integration is mandatory for CRITICAL findings and recommended for HIGH findings in regulated environments. Your harness should write findings to a review queue with the original trace span ID, the masking rule ID, the model's risk assessment, and a direct link to the trace for manual inspection. The reviewer can confirm, downgrade, or escalate the finding. Track reviewer decisions over time to measure the model's precision and to identify masking rules that generate persistent false positives. Use this feedback loop to refine the prompt's examples and constraints, but never to auto-close findings without human acknowledgment when the risk level is CRITICAL.