Mixup Calibration

Mixup constructs virtual training examples by taking linear interpolations between two random input vectors and their corresponding one-hot labels. For a pair (xᵢ, yᵢ) and (xⱼ, yⱼ), the synthetic sample is generated as x̃ = λxᵢ + (1−λ)xⱼ and ỹ = λyᵢ + (1−λ)yⱼ, where λ ~ Beta(α, α). This forces the model to output soft, linearly transitioning probabilities in the space between distinct class clusters rather than abrupt, high-confidence decision boundaries.

By training on soft targets instead of hard one-hot vectors, Mixup acts as a regularizer against overconfident predictions. The model learns that ambiguous interpolated inputs should yield proportional, mixed outputs. This directly reduces the network's tendency to assign near-100% probability to a single class when presented with inputs that lie far from the training manifold, improving the Expected Calibration Error (ECE) without requiring a separate post-hoc calibration step.

Standard empirical risk minimization encourages models to assign high confidence to training points but provides no guidance for the vast volume of off-manifold input space. Mixup extends the training distribution by populating the convex hull of the data, explicitly teaching the model to produce low-confidence, uniform-like predictions in regions between clusters. This smoothness prior is a form of vicinal risk minimization that improves generalization and calibration simultaneously.

Models trained with Mixup exhibit significantly less epistemic uncertainty collapse when encountering OOD inputs. Because the training process exposes the network to a continuous spectrum of interpolated features, the model learns not to extrapolate high-confidence predictions to anomalous inputs. This results in higher entropy outputs for OOD samples, making Mixup an effective baseline for anomaly detection and selective classification without architectural changes.

The Beta distribution parameter α controls the strength of interpolation. Values near 0 produce minimal mixing (approaching standard ERM), while large values (e.g., α > 1) concentrate λ near 0.5, creating heavy interpolation. Typical values range from 0.2 to 0.4 for image classification. Overly aggressive mixing (high α) can underfit by blurring class boundaries excessively, degrading both accuracy and calibration on the original distribution.

An extension of input-space Mixup, Manifold Mixup performs interpolation on learned hidden representations rather than raw inputs. By mixing at deeper layers of the network, it encourages smoother decision boundaries at multiple levels of semantic abstraction. This variant has been shown to produce even flatter minima in the loss landscape and further improve both classwise calibration and robustness to adversarial perturbations compared to standard input-level Mixup.

The primary metric for measuring miscalibration. ECE partitions predictions into M equally-spaced bins and computes a weighted average of the absolute difference between accuracy and confidence within each bin.

• A perfectly calibrated model has an ECE of 0.
• Mixup training directly minimizes ECE by enforcing linear behavior between samples.
• ECE is often visualized alongside a Reliability Diagram.

A post-hoc calibration method that uses a single scalar parameter T to soften or sharpen the output softmax distribution. T is optimized on a held-out validation set to minimize NLL.

• T > 1: Softens probabilities, reducing overconfidence.
• T < 1: Sharpens probabilities.
• Unlike Mixup, this is a corrective fix applied after training, not a regularizer during training.

A regularization technique that implicitly calibrates a model by preventing it from assigning full probability to the correct class. It replaces one-hot targets with a mixture of the target and a uniform distribution.

• Acts as a regularizer against overconfidence.
• Often used as a complementary technique alongside Mixup.
• Unlike Mixup, it does not create synthetic input interpolations.

Mixup primarily addresses epistemic uncertainty (model ignorance) by enforcing smooth, linear behavior in regions between training samples, reducing erratic high-confidence predictions in data-sparse areas.

• Epistemic Uncertainty: Reducible with more data or better regularization (like Mixup).
• Aleatoric Uncertainty: Irreducible noise inherent in the data (e.g., overlapping classes). Mixup cannot eliminate this.

The task of identifying inputs that are semantically different from the training data. Mixup improves OOD detection by preventing a model from assigning high confidence to nonsensical interpolations.

• A model trained with Mixup is less likely to be overconfident on OOD inputs.
• It creates a smoother decision boundary, making OOD points easier to flag via lower confidence or higher energy scores.

A loss function that is minimized only when the predicted probabilities match the true data distribution. Both Brier Score and Negative Log-Likelihood (NLL) are proper scoring rules.

• Mixup training optimizes a proper scoring rule on convex combinations of inputs, encouraging the model to output less confident, linear predictions between clusters.
• This directly incentivizes honest, calibrated forecasts.