Linkage Quality Assessment

The proportion of predicted matches that are true matches. It answers: 'Of all the pairs we linked, how many did we get right?'

• Formula: True Positives / (True Positives + False Positives)
• High Precision means a low False Match Rate
• Critical in applications where false links are costly, such as merging patient records or financial fraud detection
• A precision of 0.95 means 5% of declared matches are erroneous

The proportion of true matches that were correctly identified by the linkage algorithm. It answers: 'Of all the pairs that should have been linked, how many did we find?'

• Formula: True Positives / (True Positives + False Negatives)
• High Recall means a low False Non-Match Rate
• Essential when missing a link has severe consequences, such as failing to identify a suspect in law enforcement record linkage
• A recall of 0.90 means 10% of true matches were missed

The harmonic mean of precision and recall, providing a single balanced metric when both false matches and false non-matches carry weight.

• Formula: 2 * (Precision * Recall) / (Precision + Recall)
• Ranges from 0 to 1, with 1 representing perfect linkage
• Penalizes extreme trade-offs: a system with precision 1.0 and recall 0.1 yields an F1 of only 0.18
• The standard summary statistic reported in Privacy-Preserving Record Linkage benchmarks

The proportion of true non-matches that were incorrectly classified as matches. This is the complement of precision from the non-match perspective.

• Formula: False Positives / (False Positives + True Negatives)
• Directly quantifies the risk of erroneous data merging
• In healthcare PPRL, a false match could link two different patients' records, creating a dangerous hybrid medical history
• Often the primary constraint in regulated industries with strict data integrity requirements

The proportion of true matches that were missed by the linkage algorithm. This is the complement of recall.

• Formula: False Negatives / (False Negatives + True Positives)
• Represents the fragmentation of entity records across a database
• A high false non-match rate in a master data management context means duplicate records persist, undermining the single customer view
• Often trades off against the false match rate via match score thresholding

The process of creating a gold-standard labeled dataset against which automated linkage is evaluated. Without it, quality metrics cannot be computed.

• Involves manual adjudication by domain experts who examine record pairs
• Pairs in the uncertainty region of the match score distribution receive the most scrutiny
• For PPRL, ground truth must be established on plaintext before encoding, then mapped to cryptographic tokens
• The quality of the evaluation is bounded by the accuracy of the clerical review itself

The two foundational metrics for linkage quality. Precision measures the proportion of predicted matches that are true matches, quantifying the cost of false positives. Recall measures the proportion of true matches successfully identified, quantifying the cost of false negatives.

• Precision = True Positives / (True Positives + False Positives)
• Recall = True Positives / (True Positives + False Negatives)
• High precision minimizes incorrect record merges; high recall minimizes missed connections.
• Trade-offs between the two are managed through match score thresholding.

The harmonic mean of precision and recall, providing a single balanced metric when both false matches and false non-matches carry equal weight. The F1 score is the standard summary statistic for linkage quality.

• Formula: F1 = 2 × (Precision × Recall) / (Precision + Recall)
• Ranges from 0 to 1, with 1 representing perfect linkage.
• The F-beta variant weights recall higher than precision when false non-matches are more costly, such as in public health surveillance.

These complementary error rates decompose linkage failures into two distinct categories. The False Match Rate (FMR) is the proportion of true non-matching pairs incorrectly linked, while the False Non-Match Rate (FNMR) is the proportion of true matching pairs missed.

• FMR directly impacts data integrity by creating spurious composite records.
• FNMR leads to fragmented entity profiles and undercounting.
• In PPRL, cryptographic encoding choices influence both rates; hardened Bloom filters reduce FMR at the cost of slightly higher FNMR.

Linkage quality assessment requires a gold standard dataset where true match status is known with certainty. Construction methods include:

• Synthetic generation: Creating artificial datasets with known duplicates and injected errors.
• Clerical review: Manual adjudication by domain experts on a representative sample.
• Deterministic cross-reference: Using a high-integrity unique identifier available in a trusted third-party registry.
• The ground-truth must reflect realistic data quality issues, including typographical errors, missing values, and formatting inconsistencies.

Systematically evaluating linkage quality across a range of match score thresholds to understand the precision-recall trade-off curve. This analysis identifies the optimal operating point for a specific business context.

• A Receiver Operating Characteristic (ROC) curve plots true positive rate against false positive rate.
• A Precision-Recall curve is more informative when the class distribution is highly imbalanced, as is typical in record linkage.
• The area under the precision-recall curve (AUPRC) provides a threshold-independent quality metric.

The manual process of resolving record pairs that fall into an uncertainty region between clear match and clear non-match thresholds. Clerical review serves dual roles:

• Operational: Making final determinations on ambiguous pairs to improve linkage completeness.
• Evaluative: Providing labeled data for assessing automated classifier accuracy.
• Review interfaces typically present side-by-side record views with highlighted field-level agreement and disagreement patterns.
• Inter-rater reliability metrics, such as Cohen's kappa, measure consistency across multiple reviewers.