Entity Resolution

A rule-based approach where records are linked only when specific fields match exactly or according to predefined logical conditions. This method relies on unique identifiers or exact field comparisons, such as matching on Social Security Number or a composite key of first name, last name, and date of birth. While highly precise, deterministic matching suffers from low recall when data quality is poor, as it cannot tolerate typographical errors, transpositions, or missing values. It is best suited for high-certainty use cases where false positives are unacceptable.

A statistical framework, often based on the Fellegi-Sunter model, that calculates the likelihood that two records refer to the same entity by weighing the agreement and disagreement of multiple fields. Each field is assigned a weight based on its discriminatory power and error rate:

• Agreement weight: Log-likelihood ratio if fields match
• Disagreement weight: Log-likelihood ratio if fields differ These weights are summed to produce a composite score, which is compared against two thresholds to classify pairs as a match, non-match, or requiring clerical review.

A performance optimization technique that partitions the dataset into mutually exclusive blocks to avoid the computationally prohibitive O(n²) comparison of all possible record pairs. Common blocking keys include phonetic encodings like Soundex or Metaphone, or n-gram-based locality-sensitive hashing. Only records within the same block are compared, dramatically reducing the search space. Sophisticated implementations use multi-pass blocking across different keys to mitigate the risk of missing true matches that fall into different blocks due to data errors.

A modern approach that frames entity resolution as a binary classification problem. A training set of labeled record pairs is used to train a model—such as a random forest, gradient boosting machine, or deep neural network—to predict whether a pair is a match. Features include:

• Field-level similarity scores (Jaro-Winkler, Levenshtein, cosine similarity)
• Token-based features (TF-IDF, Jaccard index)
• Structural features (relationship context in a graph) This method automatically learns complex, non-linear interaction effects between fields that are difficult to capture with hand-tuned probabilistic weights.

The final stage of entity resolution where pairwise match decisions are consolidated into connected components or clusters, each representing a single real-world entity. A canonical record (or 'golden record') is then synthesized for each cluster by selecting the most complete, accurate, or recent value for each attribute from among the constituent records. Algorithms like Union-Find or hierarchical agglomerative clustering are used to transitively resolve matches, ensuring that if record A matches B, and B matches C, then A, B, and C all belong to the same entity cluster.

A specialized methodology for linking records across organizations without exposing sensitive personally identifiable information (PII). Techniques include:

• Bloom filters: Encode identifiers into bit arrays for secure comparison
• Homomorphic encryption: Allows computation on ciphertexts
• Secure multi-party computation: Distributed protocols where no party reveals its private input This is critical in healthcare for linking patient records across providers or in government for fraud detection, enabling entity resolution while maintaining compliance with regulations like HIPAA and GDPR.

The statistical process of identifying records across different datasets that refer to the same entity when a common unique identifier does not exist. Unlike simple deduplication, record linkage focuses on joining disparate sources.

• Deterministic linkage: Exact matching on a pre-defined set of rules or keys.
• Probabilistic linkage: Uses statistical weights (e.g., Fellegi-Sunter model) to calculate match likelihood based on agreement/disagreement patterns.
• Blocking: A performance optimization that groups similar records into candidate blocks to avoid O(n²) comparisons.

The NLP task of grounding an ambiguous textual mention (e.g., 'Apple') to its canonical entry in a knowledge base (e.g., Apple Inc. vs. the fruit). This is distinct from entity resolution, which merges records; linking connects text to a structured ID.

• Mention Detection: First identifying the text span of an entity.
• Candidate Generation: Retrieving possible KB entries via alias tables or dense retrieval.
• Disambiguation: Using context vectors to rank candidates, often with BERT-based cross-encoders.

The clinical NLP process of mapping diverse surface forms to a single canonical concept identifier in a standardized ontology like SNOMED CT or RxNorm. It resolves lexical variation (e.g., 'heart attack' → 22298006 |Myocardial infarction|).

• Handles synonymy (different names, same concept) and polysemy (same name, different concepts).
• Often uses dense vector similarity search over pre-computed concept embeddings.
• Critical for aggregating patient cohorts across different EHR systems.

The process of discovering semantic correspondences (mappings) between concepts in different ontologies. This enables interoperability when entity resolution must operate across systems using different underlying schemas.

• Equivalence: owl:sameAs or exact match.
• Subsumption: One concept is broader/narrower than another.
• Alignment APIs: Tools like the Ontology Alignment Evaluation Initiative (OAEI) provide benchmarks for automated matchers.

The atomic unit of an RDF knowledge graph, structured as (Subject, Predicate, Object). After entity resolution merges duplicate nodes, the resulting unified entity becomes the subject or object in these factual assertions.

• Example: http://dbpedia.org/resource/Marie_Curie http://dbpedia.org/ontology/birthPlace http://dbpedia.org/resource/Warsaw.
• Resolution quality directly impacts the accuracy of graph queries like SPARQL.

The Shapes Constraint Language is a W3C standard for validating RDF graphs against a set of conditions. After entity resolution merges records, SHACL ensures the resulting unified entities conform to the expected schema.

• Validates cardinality (e.g., a patient must have exactly one date of birth).
• Checks data type conformance and unique identifier constraints.
• Acts as a quality gate before resolved data enters production knowledge graphs.