GENRE (Generative Entity Retrieval)

GENRE identifies entity boundaries by generating special markup tokens around mentions within the input text.

• Start/End Tokens: Uses special tokens like [START] and [END] to demarcate entity spans
• Joint Modeling: Simultaneously detects mentions and disambiguates them in a single forward pass
• Contextual Awareness: The autoregressive nature means each generated entity conditions on previously identified entities

This eliminates the need for a separate named entity recognition pipeline.

The core innovation enabling valid entity generation is constrained beam search guided by a prefix tree.

• Trie-Guided: At each decoding step, only tokens that continue a valid entity path in the trie are allowed
• Dynamic Vocabulary Masking: The output vocabulary is dynamically masked based on the current trie node
• Guaranteed Validity: Every generated sequence is guaranteed to resolve to an existing knowledge base entry

This constraint transforms an unconstrained language model into a precise entity retrieval system.

GENRE collapses the traditional multi-stage entity linking pipeline into a single sequence-to-sequence transformer.

• No Separate Index: Unlike Bi-Encoder systems, GENRE stores entity knowledge entirely within model parameters
• Unified Training: Trained end-to-end on mention-entity pairs from Wikipedia hyperlinks
• Simplified Deployment: A single model file replaces retrieval indices, candidate generators, and cross-encoder rankers

Trade-off: Model size grows with knowledge base size, unlike retrieval-based systems that scale independently.

GENRE demonstrates strong zero-shot entity linking capabilities by leveraging the semantic knowledge encoded during pre-training.

• Cross-Lingual: Can link mentions in one language to entities described in another, using only entity descriptions as a bridge
• Unseen Entities: Generates correct entity names for entities never seen during fine-tuning by composing subword tokens
• Description Conditioning: When provided with entity descriptions as additional input, accuracy on rare entities improves significantly

This makes GENRE particularly valuable for low-resource languages and rapidly evolving knowledge bases.

GENRE can process an entire document and output a fully annotated version with all mentions linked in a single generation.

• Document-Level Context: Each entity prediction conditions on the full document context and previously linked entities
• Collective Disambiguation: Implicitly performs collective entity linking by modeling inter-entity coherence through the autoregressive factorization
• Structured Output: The generated text contains both the original content and embedded entity annotations

This holistic approach captures long-range dependencies that mention-by-mention systems miss.