Wikification

The de facto standard evaluation dataset for entity linking and wikification. It consists of Reuters newswire articles manually annotated by human judges. Key characteristics:

• Links mentions to the YAGO knowledge base, which is derived from Wikipedia.
• Divided into CoNLL (training), AIDA-A (validation), and AIDA-B (testing) splits.
• Provides a rigorous test of a system's ability to handle domain-specific news text rather than just Wikipedia articles.
• Metrics reported include micro-averaged accuracy and macro-averaged F1.

The General Entity Recognition, Disambiguation, and Linking Benchmark provides a unified framework for reproducible experiments. It eliminates the need for researchers to implement custom evaluation scripts. Core features:

• Offers a REST API where annotators can be plugged in and tested against multiple datasets.
• Standardizes output formats to compare tools like DBpedia Spotlight, WAT, and BLINK on equal footing.
• Supports multiple knowledge bases including DBpedia, Wikidata, and custom configurations.
• Reports InKB accuracy, F1, and Nil prediction metrics automatically.

An open-source, configurable tool for automatically annotating text with DBpedia URIs. It is designed for robustness and speed in production environments. Technical architecture:

• Uses a Vector Space Model combined with cosine similarity for candidate selection.
• Employs a generative probabilistic model for final disambiguation.
• Provides a Java/Scala API and a simple HTTP REST endpoint for integration.
• Supports multiple languages and can be configured to adjust the trade-off between precision and recall via confidence scores.

A state-of-the-art model from Facebook AI that performs wikification against a massive index of 5.9 million Wikipedia entities. It uses a two-stage pipeline for extreme scale:

• Stage 1 (Bi-Encoder): Independently encodes the mention context and entity descriptions into dense vectors for fast, approximate nearest neighbor retrieval using FAISS.
• Stage 2 (Cross-Encoder): A high-precision transformer that processes the concatenated mention and each top-k candidate jointly to re-rank results.
• Trained using distant supervision on Wikipedia hyperlinks, eliminating the need for expensive human annotation.

An autoregressive sequence-to-sequence model that reframes wikification as a constrained generation task. Instead of comparing dense vectors, it generates the unique Wikipedia title of the target entity token by token. Key innovations:

• Uses a BART architecture fine-tuned to output entity names.
• Employs beam search with a prefix tree (trie) constraint to ensure only valid entity names are generated.
• Naturally handles Nil prediction by generating a special token when no entity matches.
• Unifies mention detection and disambiguation into a single, end-to-end process.

The Facebook AI Similarity Search library is the critical infrastructure component enabling scalable wikification. It manages the dense embedding index for Bi-Encoder retrieval. Capabilities:

• Implements Product Quantization (PQ) and Inverted File Index (IVF) for memory-efficient, billion-scale search.
• Supports GPU acceleration for sub-millisecond query latency on high-dimensional vectors.
• Provides exact search for ground truth and approximate search for production speed.
• Used as the retrieval backbone for systems like BLINK to quickly narrow 5.9M candidates down to a manageable top-k list.