Temperature Scaling

Temperature scaling introduces a single scalar parameter T > 0 that divides all logits before the softmax function. When T > 1, the output distribution softens, increasing prediction entropy and reducing peak confidence. The optimal T is learned by minimizing the negative log-likelihood on a held-out validation set, ensuring the model's average confidence aligns with its empirical accuracy without modifying the underlying decision boundary.

A critical property of temperature scaling is its monotonicity: dividing all logits by T preserves the relative ordering of class probabilities. The predicted class label never changes, only the confidence score associated with it. This makes temperature scaling a accuracy-preserving calibration method, unlike histogram binning or isotonic regression which can alter decision boundaries and degrade classification performance.

Membership inference attacks exploit the confidence gap between training and non-training samples. Temperature scaling with T > 1 systematically reduces this gap by:

• Increasing prediction entropy on memorized training samples
• Flattening overconfident peaks that signal memorization
• Narrowing the distributional divergence between member and non-member outputs This directly degrades the signal-to-noise ratio of prediction entropy-based attacks.

Temperature scaling imposes negligible computational cost:

• Training phase: A simple 1D convex optimization over a validation set, typically converging in fewer than 50 iterations using L-BFGS
• Inference phase: A single scalar division of the logit vector, adding sub-microsecond latency per prediction
• Memory footprint: Exactly one floating-point parameter stored alongside the model This makes it suitable for deployment in latency-sensitive production environments and edge devices.

Temperature scaling assumes that the optimal T learned on a validation set generalizes to deployment data. This assumption breaks under:

• Covariate shift: When input distributions drift from training
• Open-set conditions: When novel classes appear at inference
• Adversarial querying: When attackers craft inputs specifically to probe calibration boundaries In these scenarios, T may under-correct or over-correct confidence, requiring adaptive recalibration or complementary defenses like conformal prediction.

Temperature scaling complements DP-SGD training by addressing a residual privacy leakage vector. While differential privacy bounds the influence of any single record on model parameters, it does not directly control the confidence calibration of the resulting model. Applying temperature scaling post-training:

• Reduces overconfidence on records that memorized despite DP noise
• Operates outside the privacy budget since T is learned on public validation data
• Provides a defense-in-depth layer against black-box membership inference on DP-trained models

The alignment between a model's predicted confidence and its empirical accuracy. A perfectly calibrated model that predicts 70% confidence will be correct exactly 70% of the time. Temperature scaling directly optimizes this alignment by adjusting the sharpness of the softmax distribution without affecting the rank order of predictions.

• Expected Calibration Error (ECE) quantifies the weighted average gap between confidence and accuracy across bins
• Modern neural networks tend toward overconfidence, producing probabilities that are systematically higher than their true correctness rate
• Post-hoc calibration methods like temperature scaling correct this miscalibration without retraining

A measure of uncertainty in a model's output probability distribution, calculated as the negative sum of each class probability multiplied by its logarithm. Temperature scaling directly modulates entropy—higher temperatures increase entropy, smoothing the distribution.

• Training samples typically exhibit lower entropy than non-training samples, creating the signal exploited by membership inference attacks
• Temperature scaling reduces the entropy gap between member and non-member predictions
• High entropy indicates the model is uncertain; low entropy signals confident memorization

A modeling error where the algorithm learns noise and random fluctuations in training data rather than the underlying signal. Overfitting is the root cause of the confidence gap that temperature scaling mitigates.

• Overfit models assign disproportionately high confidence to training examples while expressing uncertainty on structurally similar non-training inputs
• Temperature scaling acts as a post-hoc smoothing operator, reducing the sharp confidence peaks caused by overfitting
• Combined with weight decay and early stopping, temperature scaling provides a defense-in-depth approach to reducing membership inference leakage

An adversarial method that determines whether a specific record was in the training set by analyzing confidence scores, prediction entropy, or loss values. Temperature scaling is a primary defense because it directly reduces the signal attackers exploit.

• Attackers train shadow models on auxiliary data to learn the confidence patterns distinguishing members from non-members
• Temperature scaling increases the uncertainty overlap between training and non-training samples, degrading attack accuracy
• The defense trades a small amount of clean accuracy for a significant reduction in attack AUC

The field of estimating and representing the confidence bounds of model predictions. Temperature scaling addresses aleatoric uncertainty (inherent data noise) by recalibrating the softmax, but does not capture epistemic uncertainty (model ignorance).

• Aleatoric uncertainty: irreducible noise in the data itself, partially addressed by calibration
• Epistemic uncertainty: the model's lack of knowledge about out-of-distribution inputs, requiring Bayesian or ensemble methods
• Combining temperature scaling with Monte Carlo Dropout or deep ensembles provides a more complete UQ framework that further frustrates membership inference

A prediction strategy where the model abstains from making decisions on inputs with high uncertainty. When combined with temperature scaling, selective classification denies attackers the high-confidence signals needed for membership inference.

• Define a confidence threshold below which the model refuses to predict
• Temperature scaling smooths the confidence distribution, making it easier to set a threshold that balances coverage and risk
• Attackers querying with non-member samples receive either abstentions or low-confidence outputs, degrading their ability to train an inference classifier