Multiple Instance Learning (MIL)

In MIL, the fundamental unit of supervision is the bag—a collection of instances. The model receives a single label for the entire bag, not for its constituent parts.

• Positive bag: Contains at least one positive instance (e.g., a slide with tumor tissue)
• Negative bag: Contains only negative instances (e.g., a benign slide)
• This mirrors clinical reality where pathologists assign a single diagnosis to an entire gigapixel whole slide image without annotating every cell

The classical MIL assumption states: a bag is positive if and only if it contains at least one positive instance. This is the foundation upon which most MIL algorithms are built.

• A negative bag guarantees all instances are negative
• A positive bag reveals only that at least one instance is positive—the others may be negative or positive
• This assumption drives the max-pooling operation commonly used in MIL aggregators to extract the most suspicious instance score

Modern MIL approaches fall into two paradigms:

Instance-Level: A classifier scores each instance independently, then a permutation-invariant pooling operator (max, mean, or attention-weighted) aggregates scores into a bag prediction. Simple but can miss contextual relationships.

Embedding-Level: Each instance is mapped to a low-dimensional embedding. A permutation-invariant aggregation function—often an attention mechanism—combines these embeddings into a single bag representation before a final classifier. This preserves richer information and dominates current state-of-the-art pathology models like ABMIL and CLAM.

Introduced by Ilse et al. (2018), attention-based MIL replaces rigid pooling with a learned, weighted average of instance embeddings.

• A two-layer neural network computes an attention score for each instance
• Scores are normalized via softmax, producing a weighted sum of instance features
• The model learns to assign high attention to diagnostically relevant regions (e.g., tumor nests) and low attention to irrelevant tissue (e.g., stroma, fat)
• This provides inherent interpretability—attention maps can be visualized as heatmaps over the slide, showing which regions drove the prediction

A critical mathematical constraint in MIL: the bag-level prediction must be invariant to the order of instances. Swapping two patches must not change the output.

• This is guaranteed by using symmetric aggregation functions (sum, mean, max, attention-weighted mean)
• Operations like concatenation or recurrent neural networks violate this property unless carefully constrained
• The Deep Sets theorem (Zaheer et al., 2017) proves that any permutation-invariant function on a set can be decomposed as: ρ(Σ φ(x_i))—a sum of transformed elements followed by a final transformation

Whole Slide Image (WSI) analysis is the killer application of MIL. A single WSI can contain 100,000+ tissue patches at high magnification, making dense annotation infeasible.

• The WSI is tiled into patches (e.g., 256×256 pixels at 20× magnification)
• A pre-trained feature extractor (e.g., a Vision Transformer or ResNet) encodes each patch into a feature vector
• The MIL aggregator processes these vectors to produce a slide-level diagnosis
• Leading frameworks: CLAM (clustering-constrained attention), DSMIL (dual-stream), and TransMIL (transformer-based aggregation)

A learning paradigm where models are trained on coarse-grained or noisy labels rather than precise, instance-level annotations. In computational pathology, this means a model learns to classify an entire Whole Slide Image (WSI) as cancerous based only on the slide-level diagnosis, without requiring a pathologist to annotate every individual cell. This is the superordinate category that MIL falls under, contrasting with fully supervised learning that requires exhaustive pixel-level segmentation masks.

A permutation-invariant aggregation operator that computes a weighted average of instance embeddings, where the weights are learned by a gated attention mechanism. This allows the model to assign higher importance to diagnostically relevant patches (e.g., tumor regions) and down-weight irrelevant ones (e.g., stroma or adipose tissue). Key properties include:

• Permutation Invariance: The output is unchanged regardless of the order of patches in the bag.
• Interpretability: Attention weights can be visualized as a heatmap over the WSI to identify regions of interest.
• Trainability: The entire pipeline, from feature extractor to attention mechanism, is end-to-end differentiable.

A gigapixel digital representation of a glass pathology slide, scanned at high magnification (typically 20x or 40x). A single WSI can contain billions of pixels and is far too large to process as a single input to a GPU. In the MIL framework, the WSI is the bag, and the thousands of smaller image tiles or patches extracted from it are the instances. The inherent challenge is that a WSI-level label (e.g., 'metastatic carcinoma') applies to the entire slide, but the disease may only occupy a small fraction of the tissue.

The classical MIL framework relies on one of two assumptions that define the relationship between instance labels and bag labels:

• Standard Assumption: A bag is positive if and only if it contains at least one positive instance. This is the most common assumption in binary classification.
• Collective Assumption: All instances in a bag contribute equally to the bag label, and the bag label is determined by a summary statistic (e.g., mean, max) of instance predictions. Modern attention-based MIL relaxes these rigid assumptions by learning a flexible, weighted combination of instances.

Two fundamental approaches to implementing MIL:

• Instance-Level: A classifier scores each instance independently, and these scores are aggregated (e.g., via max-pooling) to produce a bag-level prediction. This is computationally efficient but can lose contextual information.
• Embedding-Level: Each instance is first mapped to a feature vector (embedding). These embeddings are then aggregated into a single bag-level representation, which is passed to a final classifier. This approach preserves richer information and is the dominant paradigm in modern attention-based MIL for pathology.