Entity Reconciliation

Unlike deterministic exact-match logic, reconciliation relies on probabilistic algorithms that calculate the likelihood of a match. These engines evaluate multiple attributes simultaneously:

• String Similarity: Levenshtein distance, Jaro-Winkler, and phonetic algorithms (Soundex, Metaphone) for fuzzy name matching.
• Token-Based Scoring: TF-IDF weighting on entity labels and aliases to prioritize distinctive terms.
• Geospatial Proximity: Haversine formula calculations for location-based entities with coordinate data.
• Temporal Alignment: Date range overlap analysis for entities with birth/death or founding/dissolution dates.

The engine outputs a confidence score between 0 and 1, allowing systems to set thresholds for automatic acceptance versus human review queues.

Brute-force comparison against millions of knowledge base entries is computationally prohibitive. Reconciliation systems use blocking techniques to reduce the search space:

• Attribute-Based Blocking: Grouping records by shared high-cardinality attributes like ZIP codes or industry codes before detailed comparison.
• Phonetic Blocking: Indexing entities by their Soundex or Double Metaphone encodings to cluster phonetically similar names.
• Embedding-Based Nearest Neighbor Search: Encoding entity descriptions into dense vectors and using approximate nearest neighbor (ANN) algorithms like HNSW to retrieve top-K candidates.
• Type Filtering: Restricting candidate generation to entities of the same semantic type (e.g., only matching against Q5 for humans or Q4830453 for businesses).

When multiple candidates have similar surface forms (e.g., 'Paris, Texas' vs. 'Paris, France'), reconciliation engines leverage graph topology for disambiguation:

• Relationship Fingerprinting: Comparing the set of connected entities—a person's employer, co-authors, or family members—to create a unique semantic fingerprint.
• Category Membership: Validating that the candidate belongs to expected ontological categories (e.g., instance of Q515 for cities).
• Sitelink Verification: Checking alignment with Wikipedia articles in specific languages to confirm regional relevance.
• Property Cardinality Constraints: Ensuring the candidate satisfies expected uniqueness constraints (e.g., an entity should have exactly one P569 birth date). This transforms reconciliation from a simple string-matching problem into a graph-based identity verification task.

For matches below the auto-accept threshold, reconciliation systems implement curation interfaces that present human validators with:

• Side-by-Side Comparison: Displaying local record attributes alongside candidate knowledge base properties.
• Evidence Highlighting: Visualizing which fields contributed most to the match score (e.g., exact name match vs. fuzzy date alignment).
• Override Actions: Allowing curators to confirm, reject, or create new knowledge base entries when no suitable candidate exists.
• Feedback Loops: Logging human decisions to retrain matching models and improve future automated reconciliation accuracy. This hybrid approach ensures high precision for critical entity identity decisions while maintaining throughput for bulk processing.