Attention-Based MIL

Replaces fixed pooling operators (max or mean) with a trainable attention network. The model learns to assign a weight to each instance, quantifying its contribution to the bag-level decision. This creates a permutation-invariant, differentiable operator that is optimized end-to-end.

• Two-layer neural network with tanh and sigmoid activations
• Outputs a normalized weight for every patch in the slide
• Weights sum to 1, creating a probabilistic interpretation

The attention weights can be visualized as a diagnostic heatmap overlaid on the whole slide image. High-attention regions directly correspond to areas the model found most influential for its prediction.

• Enables pathologist verification of model focus
• Identifies tumor epithelium, inflammation, or necrosis without pixel-level labels
• Transforms a black-box classifier into an auditable diagnostic tool

An extension of standard attention that introduces a gating mechanism to learn non-linear relationships between instances. The gate suppresses irrelevant features while amplifying diagnostically salient ones.

• Uses a sigmoid gate paired with a tanh activation
• Prevents saturation issues in deep networks
• Improves performance on highly heterogeneous tissue patterns

Each patch is first encoded by a pre-trained convolutional neural network (e.g., ResNet50) into a compact feature vector. The attention mechanism operates on these embeddings, not raw pixels.

• Reduces dimensionality from millions of pixels to a 1024-dimensional vector
• Enables efficient processing of gigapixel slides
• Supports transfer learning from ImageNet or pathology-specific foundation models

For multi-class diagnostic problems, the architecture can be extended with parallel attention branches. Each class learns its own set of attention weights, identifying distinct morphological regions for each diagnostic category.

• One attention branch per tumor subtype or grading level
• Enables differential diagnosis from a single slide
• Reveals class-specific morphological signatures

A regularization technique that encourages the attention network to focus on morphologically coherent tissue regions rather than scattered individual cells. This mimics how pathologists examine contiguous tissue architecture.

• Adds a spatial continuity loss to the training objective
• Reduces noise from isolated high-attention artifacts
• Produces more clinically plausible heatmaps

The preprocessing step that divides a gigapixel WSI into thousands of smaller, manageable image tiles (e.g., 256x256 pixels at 20x magnification). These patches serve as the input instances for the MIL model.

• Requires efficient tissue detection to discard empty background patches.
• Coordinates are stored to map predictions back to the original slide.
• Patch size and overlap are critical hyperparameters affecting downstream attention maps.

The visualization technique that renders the attention weights back onto the whole slide image. Each patch is color-coded by its attention score, creating a probability heatmap.

• High-attention regions (red) indicate areas the model found most diagnostically relevant.
• Provides a form of spatial explainability for pathologists.
• Used to verify the model is focusing on tissue morphology, not artifacts.

The primary downstream task enabled by attention-based MIL. The model assigns a single diagnostic label (e.g., cancerous vs. benign) to the entire gigapixel image.

• The aggregation function collapses all patch-level features into one representation.
• The final classification layer operates on this slide-level representation.
• Enables training from pathology reports without requiring pixel-level annotations.

The multi-resolution image storage structure that enables efficient navigation of WSIs. A pyramid stores the slide as a series of downsampled layers, similar to digital maps.

• The baseline layer contains the full-resolution image.
• Intermediate layers are power-of-two reductions.
• Patch extraction typically reads from a specific pyramid level (e.g., level 0 for 40x).

A pre-trained convolutional neural network or vision transformer that converts raw image patches into fixed-length feature vectors before they enter the attention MIL aggregator.

• Common backbones: ResNet-50, ViT-S/16, or pathology-specific foundation models like UNI or CONCH.
• Features are often pre-computed and stored to accelerate training.
• The backbone is typically frozen during MIL training to manage GPU memory.