Entity Linking

The initial retrieval phase that fetches a set of plausible knowledge base entries for a given mention. This step uses alias tables, surface form dictionaries, and TF-IDF or BM25 retrieval to narrow billions of entities down to a manageable shortlist. Key techniques include:

• Name-to-ID mapping from Wikipedia redirects and disambiguation pages
• Prefix-encoded indices for sub-string matching on partial mentions
• Contextual hashing using locality-sensitive hashing (LSH) for fast approximate lookup

Without efficient candidate generation, the subsequent ranking step becomes computationally intractable.

The core ranking step that scores each candidate entity against the surrounding document context. Modern systems use cross-encoders or bi-encoders with contextualized embeddings from models like BERT. The model computes a similarity score between the mention-in-context representation and the entity's canonical description. Advanced implementations incorporate:

• Entity-aware transformers like LUKE that inject knowledge graph embeddings into self-attention
• Contrastive learning objectives using InfoNCE loss to separate correct entities from hard negatives
• Global coherence models that jointly disambiguate all mentions in a document using Graph Attention Networks (GATs)

The critical ability to recognize when a mention has no corresponding entry in the target knowledge base. A high-quality entity linker must output a NIL identifier rather than forcing a false match. This is implemented via:

• A dedicated NIL threshold on the final similarity score
• An explicit NIL class in the training objective, trained on mentions known to be absent from the KB
• Out-of-domain detection using energy-based models or Mahalanobis distance in the embedding space

Nil prediction prevents hallucinated links that degrade downstream knowledge graph quality.

The prerequisite segmentation task that identifies text spans referring to entities. This is often performed by a Named Entity Recognition (NER) model that jointly predicts span boundaries and entity types. Key architectural choices include:

• Span-based architectures that enumerate and score all possible n-grams up to a maximum length
• BIO/BILOU tagging schemes for token-level classification in sequence labeling models
• Joint modeling where mention detection and entity disambiguation share a single encoder, reducing cascading errors

Accurate mention boundaries are essential; a misaligned span will almost certainly link to the wrong entity.

A constraint layer that verifies the predicted entity's semantic type aligns with the mention's expected type. For example, a mention tagged as a PER (person) should not link to a LOC (location) entity. This is enforced through:

• Type hierarchies from the knowledge base ontology (e.g., Wikidata's P31 instance-of relations)
• Vector-space type embeddings that penalize type-incompatible candidates during ranking
• Schema.org alignment for web-scale entity linking, ensuring linked entities match expected @type declarations

Type validation significantly reduces false positives in ambiguous contexts.

The capability to link mentions to entities never seen during training. This is essential for dynamic knowledge bases that continuously add new entries. Techniques include:

• Dual-encoder architectures that encode mentions and entity descriptions into a shared dense vector space
• Pre-computed entity embeddings stored in a vector database for fast approximate nearest neighbor (ANN) search
• Distant supervision to generate training data by aligning unlabeled corpora with knowledge base entries

Zero-shot linking ensures the system remains functional as the underlying knowledge graph evolves without requiring retraining.

The foundational information extraction subtask that locates and classifies named entities in text into pre-defined categories such as person, organization, location, and medical codes. NER serves as the essential precursor to entity linking by identifying the textual spans that require disambiguation. Modern approaches use transformer-based architectures with token-level BIO tagging schemes to achieve state-of-the-art performance across domains.

The NLP task of finding all expressions in a text that refer to the same real-world entity. This includes linking pronouns (he, she, it), definite noun phrases (the company, the CEO), and named mentions back to their antecedents. Effective coreference resolution dramatically improves entity linking accuracy by ensuring that all mentions of an entity—not just its first explicit name—are connected to the correct knowledge base entry.

The specific entity linking task that maps textual mentions to their corresponding Wikipedia articles. Tools like DBpedia Spotlight, TagMe, and Wikipedia Miner perform wikification by leveraging Wikipedia's extensive anchor text corpus and page link structure. This process transforms ambiguous surface forms like 'Paris' into unambiguous Wikipedia URLs, enabling downstream knowledge graph population and semantic enrichment.

A technique that creates low-dimensional vector representations of the nodes and edges in a knowledge graph while preserving structural and semantic properties. Models like TransE, RotatE, and ComplEx learn these embeddings by optimizing scoring functions over triples. These dense representations enable entity linking systems to perform candidate ranking by measuring vector similarity between mention contexts and knowledge base entity embeddings.

The process of identifying, linking, and merging records that correspond to the same real-world entity across different data sources or within a single database. Also known as record linkage or deduplication, this technique uses blocking keys, fuzzy string matching, and probabilistic models to resolve entities like 'IBM Corp.' and 'International Business Machines' into a single canonical identifier before linking to a knowledge base.

A weak supervision paradigm that automatically generates labeled training data by aligning an unlabeled text corpus with an existing knowledge base. By heuristically matching entity mentions to knowledge base entries, distant supervision creates large-scale—though noisy—training datasets for entity linking models. This approach enables training on massive corpora without expensive manual annotation, though it requires careful noise mitigation strategies.