Identity Resolution Accuracy

Precision measures the correctness of the links the system does make. It is the proportion of predicted matching pairs (resolved entities) that are true matches.

• Formula: Precision = True Positives / (True Positives + False Positives)
• Interpretation: A high precision score (e.g., 99%) indicates that when the system declares two records are the same, it is almost always correct. This minimizes false merges, which can corrupt data integrity.
• Trade-off: Optimizing for precision alone may lead to a conservative system that misses valid matches (lower recall).

Recall measures the system's ability to find all the links that should be made. It is the proportion of actual matching pairs in the dataset that the system successfully identifies.

• Formula: Recall = True Positives / (True Positives + False Negatives)
• Interpretation: A high recall score (e.g., 95%) means the system is effective at finding most true matches, minimizing missed links that leave an entity fragmented across multiple nodes.
• Trade-off: A system tuned for high recall may generate more incorrect candidate matches, lowering precision.

The F1-Score is the harmonic mean of Precision and Recall, providing a single metric that balances the two, especially useful when the class distribution is imbalanced.

• Formula: F1 = 2 * (Precision * Recall) / (Precision + Recall)
• Interpretation: An F1-score offers a unified view of a system's accuracy. A score of 0.90 indicates a strong balance between making correct links (high precision) and finding most links (high recall).
• Use Case: It is the standard metric for comparing different entity resolution algorithms or configuration tunings on a fixed benchmark.

These are the foundational calculations from which cluster-level metrics are derived. Evaluation is performed on a set of record pairs.

• True Positive (TP): A pair of records that refer to the same entity and the system correctly links them.
• False Positive (FP): A pair of records that refer to different entities, but the system incorrectly links them.
• False Negative (FN): A pair of records that refer to the same entity, but the system fails to link them.
• True Negative (TN): A pair of different entities correctly kept separate. Often vast in number and less informative.

Pairwise metrics are computationally intensive but provide the granular ground truth for model training and validation.

BCubed metrics extend pairwise evaluation to assess the quality of the final clusters of records, which is the actual output of entity resolution.

• BCubed Precision: For each record, it measures the proportion of other records in its system-generated cluster that truly belong with it.
• BCubed Recall: For each record, it measures the proportion of other records that truly belong with it that are actually in its system-generated cluster.

BCubed offers a more holistic assessment than simple pairwise metrics because it accounts for the transitive nature of clustering: if record A is linked to B, and B to C, then A and C are in the same cluster.

All accuracy metrics require a Gold Standard or ground truth dataset for calculation. This is a manually curated, expert-verified set of record pairs or clusters where the true match/non-match status is known.

• Creation: Often sampled from production data and labeled by domain experts. High Inter-Annotator Agreement (IAA) is critical for reliability.
• Usage: The gold standard is split into training (for tuning linkage rules or ML models) and test sets (for final, unbiased accuracy reporting).
• Challenge: Maintaining the gold standard's relevance as source data schemas and distributions evolve (Drift Detection) is an ongoing task for governance teams.

Entity Accuracy measures the proportion of entities in a knowledge graph that correctly correspond to their real-world referents, free from misidentification or misrepresentation. It is a foundational metric that precedes resolution accuracy.

• Direct Assessment: Often evaluated by sampling nodes and verifying them against authoritative sources.
• Prerequisite for Resolution: High entity accuracy is necessary for meaningful identity resolution, as you cannot correctly merge incorrect entities.
• Example: In a customer knowledge graph, an entity labeled "John Smith (NYC)" must refer to the actual person John Smith in New York, not a different individual or a fictional construct.

Link Validity evaluates whether the relationships (edges) between entities in a knowledge graph are semantically correct and factually accurate. After resolving identities, the correctness of the connections between the canonical entities is critical.

• Semantic Correctness: Ensures a worksFor relationship connects a person to an organization, not to another person.
• Factual Grounding: The linked fact must be true (e.g., Albert Einstein bornIn Ulm).
• Impact on Resolution: Invalid links between duplicate entity records can mislead resolution algorithms, causing false merges or missed merges.

Canonicalization Fidelity assesses how faithfully a knowledge graph's process of selecting and maintaining a single, authoritative representation for each real-world entity preserves the meaning and attributes of all source records.

• Core Process: The act of choosing the "golden record" from merged duplicates.
• Fidelity Measures: Evaluates if the canonical entity retains all true attributes from its source records and discards erroneous ones.
• Direct Relationship: This is the outcome metric for Identity Resolution Accuracy. High resolution accuracy should result in high canonicalization fidelity.

Reference Integrity is a data quality principle ensuring that every relationship in a knowledge graph points to a valid, existing target entity, preventing dangling links. It is a structural prerequisite for reliable resolution.

• Broken Link Prevention: After merging duplicate entities (Record A and Record B into Canonical Entity C), all links pointing to A or B must be correctly redirected to C.
• Impact of Failure: Dangling references (Person X manages [deleted entity]) break graph traversals and corrupt query results, undermining the utility of accurate resolution.

Rule-Based Validation is a quality assessment method that checks knowledge graph data against a set of predefined logical, syntactic, or semantic rules to identify violations. It is a primary technique for enforcing constraints that ensure resolution accuracy.

• Functional Dependency Rules: e.g., "If two records have the same government-issued unique ID, they must be merged."
• Disjointness Rules: e.g., "A Person entity cannot also be a Organization entity."
• Cardinality Rules: e.g., "A person can have only one biological mother." Violations often indicate unresolved duplicates or erroneous merges.

A Gold Standard is a curated, high-quality reference dataset, created by domain experts, used as a benchmark for training, testing, and evaluating the accuracy of processes like identity resolution.

• Evaluation Benchmark: Used to calculate Identity Resolution Accuracy metrics (Precision, Recall, F1-Score) by comparing system outputs against expert-verified truths.
• Creation: Involves manual inspection and labeling of record pairs as "Match" or "Non-Match."
• Critical Role: Without a gold standard, accuracy claims are unverifiable. It is the ground truth against which algorithmic performance is measured.