Entity Disambiguation

Disambiguation relies on Named Entity Recognition to first identify and classify text spans as entities (person, organization, location). NER models like BERT-based token classifiers label each token with BIO (Begin, Inside, Outside) tags, creating the candidate mentions that downstream disambiguation resolves. Without accurate NER, disambiguation cannot begin.

The system queries a knowledge base (e.g., Wikidata, DBpedia) to retrieve all possible identities matching a surface form. Techniques include:

• Name dictionary lookup: Pre-built mappings from surface forms to entity IDs
• Alias tables: Handling nicknames, acronyms, and transliterations
• Fuzzy string matching: Compensating for typos and spelling variants
• Prior probability ranking: Ordering candidates by commonness in a reference corpus

Disambiguation models evaluate local context windows around the mention and global document coherence. Key features include:

• Bag-of-words similarity between surrounding text and entity descriptions
• Entity type compatibility with the sentence's semantic frame
• Coherence modeling: Ensuring all resolved entities in a document relate to a consistent topic domain
• Temporal signals: Matching dates and time references to entity properties

Local disambiguation resolves each mention independently using immediate context. Collective disambiguation jointly optimizes all mentions in a document, leveraging the principle that co-occurring entities should be semantically related. Graph-based algorithms like PageRank on entity coherence graphs outperform isolated approaches by propagating disambiguation decisions through the document's entity network.

Modern systems use bi-encoder architectures that embed both mention context and entity descriptions into a shared dense vector space. The model computes cosine similarity between the mention embedding and candidate entity embeddings. Cross-encoders provide higher accuracy by jointly encoding mention-entity pairs, though at greater computational cost. Dual-encoder + re-ranker pipelines balance speed and precision.

When no knowledge base entry matches a mention, the system must predict NIL (Not In List) rather than forcing a false match. Advanced systems perform NIL clustering to group unlinkable mentions referring to the same real-world entity, building emergent knowledge base entries. This is critical for handling emerging entities, rare long-tail references, and domain-specific terminology absent from general-purpose knowledge bases.

The precursor task that identifies and classifies named entities in text into predefined categories before disambiguation occurs. NER locates the mention; disambiguation identifies which specific entity it refers to.

• Common types: Person, Organization, Location, Date, Product
• Modern approaches: Transformer-based token classification using BERT variants
• Boundary detection: Determines exact start and end of multi-word entities

NER must correctly identify 'Apple' as an organization before disambiguation can determine if it's the tech company or a fruit company.

The task of identifying all expressions in a text that refer to the same real-world entity, including pronouns, definite descriptions, and aliases. This expands the context available for disambiguation.

• Anaphora: Resolving 'she' or 'the company' back to a named mention
• Cataphora: Resolving a pronoun that precedes its referent
• Split antecedents: Handling plural references like 'they' referring to multiple entities

Example: 'Sundar Pichai announced earnings. He cited AI growth.' Coreference links 'He' to Sundar Pichai, providing richer context for disambiguation.

A structured semantic network of entities, their attributes, and relationships that serves as both the target and context source for disambiguation. Entity disambiguation populates and queries knowledge graphs.

• Nodes: Represent entities with unique identifiers
• Edges: Typed relationships like 'worksAt', 'foundedBy', 'locatedIn'
• Ontologies: Formal schemas defining entity types and relationship constraints

Knowledge graphs provide the candidate entities and contextual features—such as an entity's known associates or locations—that disambiguation algorithms use to make decisions.

The parallel task of resolving ambiguous words rather than named entities. While entity disambiguation handles proper nouns like 'Washington', WSD handles common nouns and verbs with multiple meanings.

• Polysemy: Distinguishing 'bank' (financial institution) from 'bank' (river edge)
• Homonymy: Resolving words with unrelated meanings sharing the same form
• Lesk algorithm: Classic approach using dictionary definition overlap

Both tasks are critical for semantic search: WSD clarifies intent words while entity disambiguation grounds named references.

The retrieval step that produces a shortlist of possible entities from a knowledge base for a given mention. Efficiency and recall are critical—missing the correct candidate makes disambiguation impossible.

• Name dictionaries: Pre-built mappings from surface forms to entity IDs
• Alias tables: Handle variations like 'JFK' for 'John F. Kennedy'
• Acronym expansion: Resolving 'ML' to 'Machine Learning' or 'Major League'
• Fuzzy matching: Catching typos and morphological variants

Modern systems use dense retrieval with bi-encoder models to generate candidates based on semantic similarity rather than exact string matching.