BLINK (Billion-scale Entity Linking)

BLINK decomposes entity linking into a fast candidate retrieval stage followed by a high-precision reranking stage. This architecture is the key to scaling to millions of entities without sacrificing accuracy.

• Stage 1 (Bi-Encoder): Independently encodes the mention context and each entity description into dense vectors. Uses FAISS for fast maximum inner product search to retrieve the top-k candidates from the entire knowledge base.
• Stage 2 (Cross-Encoder): Concatenates the mention and each candidate entity description, feeding them jointly through a transformer with full cross-attention. This yields a precise relevance score for final disambiguation.
• Why it works: The Bi-Encoder handles scale (billion-scale retrieval in milliseconds), while the Cross-Encoder handles ambiguity (modeling subtle interactions between mention context and entity description).

The Bi-Encoder independently encodes the mention context and each entity description into fixed-size dense vectors, enabling billion-scale retrieval via dot product similarity.

• Architecture: A dual-encoder based on BERT that produces separate embeddings for mentions and entities. No cross-attention between the two towers.
• Training: Trained with a contrastive loss (in-batch negatives) where the correct entity is scored higher than all other entities in the batch.
• Indexing: All entity embeddings are pre-computed and indexed using FAISS for exact or approximate nearest neighbor search.
• Key insight: By decoupling mention and entity encoding, the Bi-Encoder allows pre-computation of entity embeddings, making retrieval constant-time with respect to knowledge base size.

The Cross-Encoder performs joint encoding of the mention and a single candidate entity, using full cross-attention to model fine-grained semantic interactions.

• Input: The concatenation of the mention context and the entity description, separated by a special [SEP] token.
• Output: A single scalar relevance score representing the likelihood that the candidate is the correct entity.
• Why it's precise: Unlike the Bi-Encoder, the Cross-Encoder allows every token in the mention to attend to every token in the entity description, capturing subtle disambiguation signals.
• Computational cost: Too expensive to run over all entities, which is why it only reranks the top-k candidates from the Bi-Encoder (typically k=10 or k=64).

BLINK represents each knowledge base entity by encoding its textual description (title + first paragraph from Wikipedia) rather than relying solely on structured triples or graph embeddings.

• Description format: The entity title and its Wikipedia abstract are concatenated and encoded by the Bi-Encoder's entity tower.
• Zero-shot capability: Because entities are represented by natural language descriptions, BLINK can link to entities never seen during training—a property called zero-shot entity linking.
• Nil prediction: If no candidate scores above a learned threshold, BLINK predicts that the mention has no corresponding entity in the knowledge base, correctly handling out-of-KB (OOKB) mentions.
• Pre-computation: All entity embeddings are computed offline and stored in a FAISS index, enabling fast retrieval without re-encoding at inference time.

BLINK is trained using distant supervision from Wikipedia hyperlinks, automatically generating large-scale training data without manual annotation.

• Data generation: Each Wikipedia hyperlink provides a mention (the anchor text) and a ground-truth entity (the linked page). The surrounding paragraph serves as context.
• Scale: This yields millions of training examples across diverse domains, far exceeding manually annotated datasets like AIDA CoNLL-YAGO.
• Negative sampling: During Bi-Encoder training, other entities in the same batch serve as in-batch negatives, creating an efficient contrastive learning signal.
• Hard negative mining: The Cross-Encoder is fine-tuned using the top-ranked candidates from the Bi-Encoder as hard negatives, teaching it to distinguish between highly confusable entities.

BLINK leverages FAISS (Facebook AI Similarity Search) to index and query millions of entity embeddings with sub-millisecond latency.

• Index construction: Pre-computed entity embeddings are added to a FAISS index using exact search (IndexFlatIP) or approximate search (IndexIVFFlat with product quantization) for larger scales.
• Retrieval: At inference, the mention embedding queries the FAISS index to retrieve the top-k entities by inner product similarity.
• Scalability: FAISS supports GPU-accelerated search and compressed indexes, enabling BLINK to scale to knowledge bases with tens of millions of entities.
• Production readiness: The combination of pre-computed embeddings and FAISS indexing makes BLINK suitable for real-time applications requiring entity linking over large knowledge bases.