Record Linkage

A rule-based approach requiring exact, character-for-character agreement on a predefined set of patient identifiers. A match is declared only when all fields in the matching key—such as a composite of Social Security Number, date of birth, and gender—are identical across records.

• Strength: High precision with very low false-positive rates when identifiers are clean.
• Weakness: Brittle in the face of data entry errors, typos, or missing values; a single transposed digit causes a missed match.
• Use Case: Ideal for high-integrity datasets like national immunization registries where unique identifiers are rigorously validated.

A statistical framework that calculates the likelihood ratio that two records belong to the same entity, even when data contains inconsistencies. The algorithm assigns agreement weights to matching fields and disagreement weights to mismatches, summing them into a composite score.

• Fellegi-Sunter Model: The foundational mathematical model that classifies record pairs as matches, non-matches, or potential matches based on threshold cutoffs.
• Blocking: A preprocessing step that partitions the dataset into mutually exclusive blocks to avoid the computationally prohibitive comparison of all possible record pairs.
• Use Case: Essential for Master Patient Index (MPI) systems merging records from facilities with inconsistent data entry practices.

The computational strategy that makes record linkage feasible at scale by drastically reducing the comparison space. Without blocking, linking 1 million records to another 1 million requires 1 trillion pairwise comparisons.

• Traditional Blocking: Groups records that share an identical blocking key, such as phonetic encoding of last name or zip code, and only compares pairs within the same block.
• Sorted Neighborhood: Sorts records by a key and slides a fixed-size window over the sorted list, comparing only records within the window.
• Canopy Clustering: Uses a cheap, approximate distance metric to create overlapping clusters, enabling fuzzy blocking for probabilistic pipelines.

Algorithms that encode names into phonetic hashes to enable fuzzy matching that tolerates spelling variations and typographical errors. These are critical for blocking and comparison in patient identity resolution.

• Soundex: The oldest algorithm, encoding names into a letter followed by three digits. Retains the first letter and maps subsequent consonants to numeric codes.
• Metaphone & Double Metaphone: More sophisticated algorithms that account for English pronunciation rules, producing variable-length keys. Double Metaphone generates both a primary and alternate encoding.
• Use Case: Matching 'Smith' to 'Smyth' or 'Katherine' to 'Catherine' when exact matching fails due to historical data entry inconsistencies.

Mathematical functions that quantify the similarity between two strings, enabling fine-grained field-level comparisons beyond binary exact-match logic.

• Levenshtein Distance: Counts the minimum number of single-character edits—insertions, deletions, or substitutions—required to transform one string into another.
• Jaro-Winkler: Measures character transpositions and applies a prefix bonus, making it particularly effective for short strings like personal names.
• Cosine Similarity with TF-IDF: Treats strings as vectors in a high-dimensional space, measuring the cosine of the angle between them. Effective for comparing longer text fields like addresses.
• Use Case: Assigning a similarity score of 0.92 to '123 Main St' vs '123 Main Street' to feed into probabilistic weight calculations.

The operational workflow for handling record pairs that fall into a gray zone between clear matches and clear non-matches. Probabilistic systems define two thresholds to create a triage pipeline.

• Upper Threshold: Pairs scoring above this are automatically linked as matches.
• Lower Threshold: Pairs scoring below this are automatically classified as non-matches.
• Clerical Review Queue: Pairs falling between the thresholds are routed to a human-in-the-loop interface for manual adjudication by data stewards.
• Iterative Tuning: Thresholds are adjusted based on operational feedback to balance the cost of manual review against the risk of false positives and false negatives.

A record linkage technique requiring an exact, character-for-character match on a predefined set of patient identifiers. This rule-based approach typically uses a composite key—such as Social Security Number combined with Date of Birth—to establish identity. Deterministic algorithms are highly precise but brittle; a simple typo or transposition in the matching fields will cause a false negative. In healthcare, this method is often the first pass in a Master Patient Index (MPI) before escalating to probabilistic methods for ambiguous cases.

A statistical approach that calculates the likelihood ratio that two records belong to the same patient, even when data contains inconsistencies. The algorithm assigns agreement and disagreement weights to each field comparison—for example, a match on a rare surname carries more weight than a match on gender. Key techniques include:

• Fellegi-Sunter model: The foundational mathematical framework for probabilistic record linkage
• Expectation-Maximization (EM): Used to estimate matching parameters without labeled training data
• Blocking: Reduces the computational search space by grouping records on high-confidence attributes before pairwise comparison

A centralized database that maintains a unique, persistent identifier for every patient across a healthcare organization. The MPI links disparate medical records from multiple source systems—EHRs, lab systems, radiology archives—to prevent duplicate entries and ensure accurate patient identification. An Enterprise Master Patient Index (EMPI) extends this capability across an entire health system or Health Information Exchange (HIE), cross-referencing identifiers from internal and external sources. The MPI is the operational consumer of record linkage algorithms, continuously resolving identity as new records arrive.

Optimization techniques that prevent the computationally prohibitive task of comparing every record pair in large datasets. Blocking partitions records into mutually exclusive groups using high-confidence attributes—such as phonetic encoding of surnames (Soundex, NYSIIS) or ZIP code—so that only records within the same block are compared. Sorted neighborhood indexing slides a window over sorted records to capture near-matches that blocking might miss. Effective blocking is critical for scaling record linkage to millions of patient records while maintaining sub-second query performance.

The essential preprocessing step that normalizes raw patient data before linkage algorithms execute. Without standardization, even deterministic matching fails. Common operations include:

• Address normalization: Parsing and formatting street addresses to USPS standards
• Name parsing: Decomposing full name strings into given, middle, and family name components
• Date standardization: Converting heterogeneous date formats to a canonical representation
• Nickname resolution: Expanding diminutives using lookup tables (e.g., 'Bill' → 'William')
• Phone and identifier formatting: Stripping punctuation and enforcing consistent patterns

Two related but distinct identity resolution tasks. Duplicate detection (or deduplication) identifies multiple records for the same patient within a single database—a common problem when a patient is registered multiple times at the same facility. Record linkage connects records for the same patient across different databases—for example, linking a hospital's EHR data with an external lab's results feed. Both rely on the same underlying matching algorithms, but record linkage must additionally contend with schema heterogeneity and the absence of a shared primary key across source systems.