Few-Shot Relation Extraction

Transforms relation extraction into a cloze-style task by reformulating the input as a natural language prompt. The model predicts a masked token or generates a verbalizer that maps to a relation class.

• Example prompt: "[E1] is the CEO of [E2]" → predicts "True" or "False"
• Leverages pre-trained language model knowledge acquired during pre-training
• Eliminates the need for a task-specific classification head
• Enables zero-shot transfer when relation descriptions are provided as prompts

A metric-based meta-learning approach that computes a prototype vector for each relation class by averaging the embeddings of the few available support examples. New instances are classified by measuring distance to these prototypes.

• Support set: K labeled examples per relation type
• Query instance: Classified by nearest prototype in embedding space
• Distance metrics commonly used: Euclidean or cosine similarity
• Effective when intra-class variance is low and prototypes are well-separated

A neural attention mechanism over the support set that classifies query instances by comparing them directly to each labeled example. Unlike prototypical networks, it does not compress the support set into a single vector.

• Uses an attention kernel to compute similarity between query and each support instance
• Preserves fine-grained information from individual examples
• Particularly effective when support examples are diverse
• Often combined with full-context embeddings where support set instances attend to each other

A tokenization strategy that inserts special reserved tokens around entity mentions to explicitly signal their positions to the model. This technique significantly improves few-shot performance by providing clear structural cues.

• Entity start/end markers: [E1_start] ... [E1_end]
• Typed markers can encode entity types: [PER_start] ... [PER_end]
• Prevents the model from needing to infer entity boundaries from raw text
• Standard practice in models like BERT-EM and MTB (Matching the Blanks)

A pre-training objective designed specifically for relation extraction where entity mentions are replaced with blank tokens. The model learns to predict whether two sentences containing the same entity pair express the same relation.

• Training signal: Binary classification — same relation or not
• Does not require any relation labels during pre-training
• Learns relation-aware representations from raw text alone
• Fine-tuned with few-shot examples for downstream relation types
• Demonstrated strong results on FewRel and TACRED benchmarks

A self-supervised learning paradigm that pulls together representations of sentence pairs expressing the same relation while pushing apart those expressing different relations. This creates a well-structured embedding space for few-shot adaptation.

• Positive pairs: Sentences with the same entity pair and relation
• Negative pairs: Sentences with different relations or entity pairs
• Uses NT-Xent loss (normalized temperature-scaled cross-entropy)
• Enables strong few-shot generalization by learning relation-invariant features
• Foundation for models like CP (Contrastive Pre-training) for relation extraction