Few-Shot Learning

The standard evaluation protocol for few-shot learning. An N-way K-shot task requires the model to classify N novel classes using only K labeled examples per class.

• Episodic training: Models are trained on a distribution of small tasks, not individual samples
• Support set: The K labeled examples provided for adaptation
• Query set: Unlabeled examples the model must classify after seeing the support set
• Medical example: A 5-way 1-shot task might require distinguishing 5 rare skin lesion types with only 1 annotated image each

A metric-based few-shot learning architecture that learns an embedding space where class prototypes can be computed from the support set. Classification is performed by finding the nearest prototype for each query sample.

• Each class prototype is the mean embedding vector of its support examples
• Distance is typically measured using Euclidean distance in the learned metric space
• Excels when intra-class variation is low and support examples are representative
• Medical application: Rapidly adapting to new cell morphology classes in digital pathology with minimal annotation

A framework that combines attention mechanisms with external memory to compare query samples directly against a support set. The model learns a similarity function rather than a fixed embedding.

• Uses cosine similarity with a learned attention kernel to weight support examples
• Naturally handles variable support set sizes without architectural changes
• Employs full context embeddings where support and query samples are encoded bidirectionally
• Well-suited for medical imaging tasks where support examples may be heterogeneous, such as rare disease identification across different imaging protocols

An optimization-based approach that learns an initialization of model parameters that can be rapidly fine-tuned to new tasks with only a few gradient steps. MAML is model-agnostic and works with any architecture trained via gradient descent.

• Inner loop: Task-specific adaptation using the support set, typically 1-5 gradient steps
• Outer loop: Meta-optimization across many tasks to find a sensitive initialization
• Learns representations that are transferable rather than task-specific
• Medical relevance: Enables rapid deployment of diagnostic models to new hospital sites with different scanner characteristics using minimal local data

A twin-network architecture that learns a similarity metric between pairs of inputs. For one-shot learning, the model compares a single support example against a query to determine if they belong to the same class.

• Two identical subnetworks process the support and query images in parallel
• The absolute difference between feature vectors is fed to a sigmoid classifier
• Trained on verification tasks rather than classification, making it robust to novel classes
• Medical use case: Verifying whether a new retinal scan matches a known rare condition from a single reference image in an atlas

An inference-time strategy that leverages the entire query set jointly, rather than classifying each query sample independently. Transductive methods use the unlabeled queries to refine decision boundaries.

• Label propagation: Constructs a graph between support and query samples to spread label information
• Transductive fine-tuning: Updates batch normalization statistics or applies entropy minimization on the query batch
• Often significantly outperforms inductive methods on medical benchmarks
• Practical scenario: When a radiologist provides 3 annotated examples of a new finding, the model can use the remaining unlabeled scans from the same batch to improve classification accuracy

A 'learning to learn' paradigm that trains a model across a distribution of tasks so it can rapidly adapt to a novel diagnostic class with only a few gradient steps. In medical imaging, meta-learning frameworks like Model-Agnostic Meta-Learning (MAML) and Prototypical Networks are foundational to few-shot learning, optimizing for an initialization that generalizes quickly to rare pathology detection where annotated samples are scarce.

A metric-based few-shot architecture that computes a prototype embedding for each class by averaging the feature vectors of the few available support examples. Classification is performed by finding the nearest class prototype in the learned metric space. This approach is highly effective for rare disease identification in dermatology and radiology, where a single prototype can represent a pathology from just 1-5 annotated scans.

A twin-branch architecture that learns a similarity function between pairs of inputs rather than explicit class boundaries. During few-shot inference, a query image is compared against each support example, and the class of the most similar support image is assigned. Siamese networks excel in one-shot medical verification tasks, such as confirming whether a retinal scan matches a known disease pattern from a single reference image.

A critical companion to few-shot learning that artificially expands the limited support set by applying label-preserving transformations. In medical imaging, this includes:

• Geometric transforms: rotation, scaling, elastic deformation
• Intensity transforms: contrast adjustment, gamma correction
• Synthetic generation: GAN-based creation of rare pathology samples Effective augmentation prevents overfitting when training on as few as 5-10 annotated examples per class.

A self-supervised pre-training strategy that learns discriminative visual representations by pulling augmented views of the same image together in embedding space while pushing different images apart. When combined with few-shot fine-tuning, contrastively pre-trained backbones like SimCLR or MoCo provide superior initialization for medical imaging tasks, enabling effective adaptation from minimal labeled data without requiring large annotated datasets upfront.

The statistical mismatch between the source domain (where the model was pre-trained or meta-trained) and the target domain (the specific medical imaging application). Few-shot learning is particularly vulnerable to domain shift because the limited support set may not fully represent the target distribution. Techniques like test-time adaptation and domain-adversarial training are often combined with few-shot methods to mitigate this degradation in clinical deployment.