Prompt-Based NER

Entity extraction is guided by a natural language template containing a [MASK] token. The model fills the mask with the entity string, converting NER into a text generation problem.

• Example Template: "[ORG] developed the product." → Input: "[MASK] developed the product."
• Decoding: The model predicts the token sequence for the masked span directly.
• Flexibility: Templates can be designed per entity type, enabling fine-grained control.

This paradigm directly exploits the parametric knowledge stored in a language model's weights during pre-training, bypassing the need for a randomly initialized classification layer.

• Few-Shot Capability: Requires significantly fewer labeled examples than traditional fine-tuning.
• Contextual Understanding: The model uses its deep linguistic understanding to disambiguate entities based on the prompt context.
• Zero-Shot Potential: With well-designed prompts, models can extract entity types never seen during fine-tuning.

Prompt-based NER unifies extraction with other NLP tasks under a single text-to-text generative paradigm, simplifying the model architecture.

• No Separate Heads: Eliminates the need for task-specific layers like CRF or linear classifiers.
• Encoder-Decoder Models: Naturally suited for models like T5, which are pre-trained for span infilling.
• Decoder-only Models: Can be adapted for autoregressive generation of entity strings in larger language models.

Prompting naturally accommodates nested and overlapping entities without requiring complex span enumeration or graph-based decoding.

• Sequential Generation: Entities can be generated one after another in a structured output format.
• Structured Prompts: Templates can instruct the model to output JSON or marked spans, handling multiple entities in a single pass.
• Disambiguation: The model resolves boundary conflicts using its internal language understanding rather than heuristic rules.

Contrasts sharply with the standard token classification approach used by models like BERT-NER.

• Traditional: Token-level BIO tagging with a linear classifier + CRF.
• Prompt-Based: Span-level generation via a masked language model head.
• Data Efficiency: Prompt-based methods often achieve higher F1 scores in low-resource settings.
• Schema Flexibility: Changing entity types only requires modifying the prompt text, not retraining a classification head.

Performance is highly dependent on the design of the prompt template and the mapping of model outputs to entity labels via a verbalizer.

• Template Search: Automated methods explore the space of possible prompt phrasings to find optimal performance.
• Verbalizer: A function that maps predicted tokens (e.g., 'Apple') to the entity class (e.g., ORG).
• Calibration: Requires careful tuning to ensure the model's output distribution aligns with the extraction task.

A machine learning paradigm where a model generalizes from only a small number of labeled examples per entity type. Prompt-based NER is a natural fit for this setting, as the template itself acts as a powerful inductive bias.

• Uses 1 to 50 examples per entity class
• Relies on pre-trained knowledge in large language models
• Evaluated on benchmarks like Few-NERD

A Machine Reading Comprehension framework that reformulates entity extraction as a question-answering task. Instead of a template with a mask, the model answers type-specific natural language questions like 'Which person is mentioned?'.

• Leverages question-answer pairs for supervision
• Naturally handles nested and overlapping entities
• Uses span extraction heads similar to SQuAD

A NER paradigm that directly enumerates and classifies arbitrary text spans as entities, bypassing the need for token-level BIO tagging schemes. Prompt-based methods often implicitly perform span categorization by generating the full entity string.

• Scores all possible start-end token pairs
• Avoids label inconsistency problems of sequence labeling
• Architectures include Global Pointer and biaffine classifiers

The task of assigning very specific semantic types from a large, hierarchically organized ontology. Prompt-based NER excels here because the template can naturally encode the type hierarchy in its instructions.

• Moves beyond coarse types like PERSON or LOCATION
• Uses types like 'American jazz musician' or 'Fortune 500 company'
• Benchmarked on datasets like Ultra-Fine Entity Typing

A method for automatically generating noisy labeled training data by aligning a text corpus with an existing knowledge base. Prompt-based NER can use these weak labels for pre-training before fine-tuning on clean data.

• Matches entity mentions to Wikipedia or Wikidata entries
• Produces large-scale but imperfect annotations
• Often combined with denoising techniques

A programmatic approach to generating training labels using multiple noisy heuristic functions, such as pattern matching and gazetteers, managed by a generative model like Snorkel. Prompt-based NER can serve as one of the labeling functions in a weak supervision pipeline.

• Combines regex, dictionaries, and heuristics
• Uses a label model to resolve conflicts between sources
• Dramatically reduces manual annotation cost