Linear Evaluation Protocol

The core principle is weight freezing. After self-supervised pre-training, all layers of the backbone encoder are locked and treated as a static feature extractor. No gradients flow backward through the backbone during evaluation.

• No fine-tuning: The backbone weights remain exactly as they were after pre-training
• Isolates representation quality: Prevents the downstream task from altering the learned features
• Standardized comparison: Ensures all methods are evaluated on equal footing, comparing only the quality of the frozen representations

A linear probe—a single fully-connected layer with no hidden units and no non-linear activation—is attached to the frozen backbone. This architectural simplicity is intentional.

• Linear separability test: If features can be separated by a hyperplane, the representations are well-structured
• No additional capacity: Prevents the classifier from compensating for poor representations with learned non-linearities
• Common implementations: Typically a nn.Linear(feature_dim, num_classes) layer in PyTorch, trained with cross-entropy loss

Reproducibility demands a rigorous, fixed recipe. The community has converged on specific hyperparameters to eliminate training tricks from influencing the benchmark.

• SGD optimizer with momentum (0.9) and no weight decay on the linear layer
• Learning rate schedule: Step decay or cosine annealing, typically trained for 90-100 epochs
• Data augmentation: Only standard augmentations (random resized crop, horizontal flip) applied during training; center crop at test time
• Batch normalization: Often applied to features before the linear classifier to stabilize training

Performance is reported using Top-1 accuracy (primary metric) and Top-5 accuracy on the validation set of standard benchmarks like ImageNet-1K.

• Top-1: The percentage of predictions where the single highest-probability class matches the ground truth
• Top-5: The percentage where the correct class is among the five highest-probability predictions
• Medical imaging variants: For medical tasks, metrics shift to AUC-ROC, sensitivity, and specificity, but the frozen linear probe methodology remains identical

The ultimate test is comparison to fully supervised pre-training. A ResNet-50 or ViT trained with full labels on ImageNet serves as the upper-bound reference.

• Closing the gap: The primary narrative in self-supervised learning papers is how close linear evaluation accuracy gets to the supervised baseline
• Surpassing supervised: Methods like DINOv2 and MAE have demonstrated that self-supervised features can exceed supervised pre-training on transfer tasks
• Medical context: In medical imaging, the baseline is often a model pre-trained on ImageNet and fine-tuned, with linear evaluation revealing whether domain-specific self-supervised pre-training adds value

Linear evaluation extends beyond the pre-training dataset to measure generalization and robustness. The frozen backbone is tested on out-of-distribution datasets without any adaptation.

• Multi-dataset benchmarking: Features are evaluated on classification tasks across different domains (e.g., from natural images to medical X-rays)
• Few-shot variants: Linear probes trained on only 1%, 5%, or 10% of labeled data test sample efficiency
• Medical imaging relevance: A backbone pre-trained via self-supervision on chest X-rays can be linearly evaluated on CT scans to assess cross-modality transfer without fine-tuning

A self-supervised paradigm that trains encoders to pull positive pairs together and push negative pairs apart in embedding space. Methods like SimCLR and MoCo rely on this principle to learn invariances to data augmentation. The quality of these representations is directly measured by the Linear Evaluation Protocol, where a frozen backbone's ability to linearly separate classes indicates the semantic structure learned during the contrastive pre-training phase.

A small multi-layer perceptron (MLP) attached to the encoder backbone during self-supervised pre-training. It maps representations to a lower-dimensional space where the contrastive loss is applied. Critically, the projection head is discarded before linear evaluation; the frozen backbone's output features are used directly to train the linear classifier, as they retain more general visual information than the projection head's loss-optimized space.

A slowly evolving copy of the primary network, updated via an exponential moving average (EMA) of the online network's weights. Used in frameworks like MoCo and BYOL, the momentum encoder provides stable, consistent target representations. During linear evaluation, the frozen momentum encoder often serves as the feature extractor, as its temporal ensembling effect produces higher-quality, less noisy representations for the downstream linear classifier.

A critical failure mode where the encoder produces a constant, non-informative vector for all inputs, achieving a trivial solution to the pre-training objective. Methods like Barlow Twins, VICReg, and BYOL employ explicit mechanisms—such as feature decorrelation, variance regularization, and stop-gradient operations—to prevent collapse. The Linear Evaluation Protocol serves as the definitive test to detect collapse, as a collapsed model will perform no better than random chance on the downstream classifier.

The process of evaluating pre-trained representations on a labeled task of interest. The Linear Evaluation Protocol is the most standardized form of transfer evaluation, isolating the representation quality from the complexities of fine-tuning. Key metrics include:

• Top-1 Accuracy: The percentage of test samples correctly classified by the linear probe.
• Top-5 Accuracy: The percentage of samples where the correct label is among the top 5 predictions.
• Few-shot performance: Accuracy when the linear classifier is trained on a very limited number of labeled examples per class.

A noise-contrastive estimation objective that maximizes the mutual information between positive pairs by framing representation learning as a categorical classification problem. The loss function is: -log(exp(sim(q,k+)/τ) / Σ exp(sim(q,k)/τ)), where τ is a temperature parameter. Models pre-trained with InfoNCE, such as SimCLR and CPC, are standard subjects of linear evaluation, as the loss directly encourages a feature space where classes are linearly separable.