Virtual Logit Matching (ViM)

ViM operates by decomposing the penultimate feature space into a principal subspace and a residual subspace. The principal subspace captures the high-variance directions corresponding to class-discriminative information, while the residual subspace captures the remaining low-variance components. OOD inputs tend to have larger norms in this residual space, providing a strong separation signal.

• Uses PCA on training features to define subspaces
• Residual norm is class-agnostic and complementary to logits
• Avoids interference with the model's learned class boundaries

The ViM score fuses two complementary signals into a single detection metric:

• Virtual Logit: The norm of the feature projection onto the residual subspace, scaled by a learned parameter
• Standard Logits: The original class logits from the model's output layer
• Combination: The virtual logit is appended as an additional dimension to the softmax, creating an implicit 'OOD class'

This dual approach captures both semantic anomalies (via logits) and distributional shifts (via residual norms).

ViM requires no model retraining or fine-tuning, making it ideal for production systems where modifying the base model is impractical. The method only needs:

• A forward pass through the trained model to extract features
• A PCA decomposition computed once on a clean in-distribution validation set
• A small calibration set to learn the scaling factor for the virtual logit

This post-hoc nature preserves the original model's accuracy while adding OOD detection capabilities.

Given a feature vector f from the penultimate layer:

1. Compute the residual: f_residual = f - W_pca · W_pca^T · f
2. Calculate the virtual logit: v = α · ||f_residual||
3. Append v to the original logit vector before softmax

The scaling factor α is calibrated to match the energy scale of the original logits, ensuring the virtual logit competes appropriately with class logits during softmax normalization.

ViM's residual norm signal is orthogonal to logit-based methods like MSP and Energy, making it highly complementary:

• MSP + ViM: Residual norm catches inputs where the model is confidently wrong
• Energy + ViM: Combines density estimation with feature-space geometry
• ReAct + ViM: Activation clipping and residual projection address different failure modes

Ensembling ViM with other post-hoc detectors often yields state-of-the-art results on benchmarks like OpenOOD.

ViM demonstrates strong performance across diverse OOD benchmarks:

• Near-OOD detection: Effectively separates semantically similar but distinct classes (e.g., dog vs. wolf)
• Far-OOD detection: Excels at rejecting inputs from entirely different domains (e.g., natural images vs. medical scans)
• Computational overhead: Minimal—only requires one PCA projection and a norm calculation per sample
• Calibration: The virtual logit naturally calibrates with temperature scaling for production thresholds