PHI Detection

PHI detection is fundamentally a token-level sequence labeling task. Modern systems use transformer-based architectures fine-tuned on clinical corpora to classify each token as the beginning, inside, or outside of a PHI entity using BIO tagging (Begin, Inside, Outside). Unlike simple regex, these models resolve ambiguity through bidirectional context.

• Example: Distinguishing 'Huntington' as a disease vs. a patient name based on surrounding syntax
• Key mechanism: Contextual embeddings capture semantic relationships that pattern-matching alone cannot resolve

A high-precision preprocessing layer applies pattern-matching rules to catch structured identifiers before they reach the ML model. This layer handles formats with strict regular expression signatures, ensuring zero false negatives for known patterns.

• Targets: Social Security numbers (\d{3}-\d{2}-\d{4}), MRNs with institutional prefixes, phone numbers, and ZIP codes
• Advantage: Deterministic rules provide guaranteed recall for structured identifiers, reducing reliance on probabilistic models for high-risk PHI categories

Production PHI detectors classify spans into 18 distinct Safe Harbor categories, not just a binary PHI/non-PHI label. Granular classification enables category-specific redaction policies and supports compliance auditing.

• Core classes: PATIENT, DOCTOR, DATE, AGE, ID, HOSPITAL, STREET, CITY, STATE, ZIP, PHONE, EMAIL, URL, SSN, MRN, ACCOUNT, DEVICE, BIOID
• Operational benefit: Allows selective retention of Limited Data Set elements (e.g., dates) when a Data Use Agreement is in place

Every PHI prediction carries a confidence score that determines its downstream routing. High-confidence predictions proceed to automated redaction, while low-confidence spans are queued for human review.

• Triage logic: Predictions below a configurable threshold (typically 0.85-0.95) are flagged for human-in-the-loop verification
• Trade-off: Lower thresholds increase recall but burden reviewers; higher thresholds risk residual PHI
• Metric: This architecture directly optimizes the false negative rate—the critical safety metric in de-identification

PHI detection extends beyond single-document analysis to cross-document coreference resolution. The system must recognize that 'the patient' in a progress note and 'John Doe' in a discharge summary refer to the same individual to apply consistent pseudonymization.

• Technique: Entity linking across documents using consistent pseudonym mapping tables
• Challenge: Maintaining referential integrity when the same real-world entity appears under different surface forms (e.g., 'Mr. Smith', 'Patient Smith', 'John S.')

PHI detection extends to pixel-level analysis of medical images. Burned-in PHI—text rendered directly into image frames—requires optical character recognition (OCR) integrated into the detection pipeline.

• Source: Ultrasound frames, scanned documents, and DICOM images with overlay text
• Pipeline: OCR extracts text regions → NLP model classifies extracted spans → redaction applied to pixel regions
• Risk: Burned-in PHI is a common source of residual PHI in imaging datasets when OCR steps are omitted