Temperature Scaling

Temperature scaling introduces exactly one learnable parameter (T > 0) that divides all logits before the SoftMax function. This parameter is optimized on a held-out validation set using negative log-likelihood as the objective. Unlike more complex calibration methods such as Platt scaling or isotonic regression, temperature scaling does not alter the rank order of predictions—the class with the highest logit remains the winner. The optimal T is found through gradient descent, making it computationally trivial to implement on top of any pre-trained neural network.

A critical property of temperature scaling is that it never changes the model's classification accuracy. Because the scalar T is applied uniformly to all logits, the relative ordering of class scores remains identical. The argmax prediction before and after scaling is mathematically guaranteed to be the same. This makes temperature scaling uniquely safe for deployment—it improves the reliability of confidence estimates without risking a degradation in hard prediction performance. For applications like open set recognition, this means the rejection threshold can be tuned on calibrated probabilities without sacrificing known-class accuracy.

Modern deep neural networks tend to produce overconfident probabilities—a model may assign 99.9% confidence to a prediction that is correct only 80% of the time. Temperature scaling directly addresses this miscalibration:

• T > 1: Softens the SoftMax distribution, reducing peak confidence and spreading probability mass across classes
• T < 1: Sharpens the distribution, making the model more confident
• T = 1: Recovers the original uncalibrated probabilities In practice, T typically converges to a value greater than 1, correcting the overconfidence bias inherent in deep classifiers trained with cross-entropy loss.

The effectiveness of temperature scaling is measured using Expected Calibration Error (ECE), which quantifies the mismatch between predicted confidence and empirical accuracy. ECE partitions predictions into bins by confidence level and computes the weighted average of the difference between accuracy and confidence within each bin. Temperature scaling consistently reduces ECE by 50-90% across standard architectures like ResNet and DenseNet, often achieving near-perfect calibration on in-distribution test data. This dramatic improvement requires only a few hundred validation samples and converges in seconds.

Temperature scaling is often confused with the temperature hyperparameter used in knowledge distillation, but they serve different purposes:

• Calibration temperature: A learned parameter optimized on a validation set to align confidence with empirical accuracy. Applied only at inference time.
• Distillation temperature: A manually set hyperparameter used during training to soften teacher model outputs and reveal inter-class relationships to the student model. In calibration, the temperature is a corrective factor applied post-training. In distillation, it is a training-time knob that controls the softness of supervisory signals.

While temperature scaling excels at calibrating in-distribution predictions, it offers no robustness to distributional shift. When a model encounters inputs from a novel modulation scheme or a different signal-to-noise ratio regime, the calibrated probabilities may remain overconfident for out-of-distribution samples. Temperature scaling does not increase the separability between known and unknown classes—it only recalibrates the confidence of predictions the model already makes. For open set recognition, temperature scaling should be combined with dedicated out-of-distribution detection methods such as ODIN or energy-based models that explicitly model epistemic uncertainty.

The post-processing or training technique of aligning a model's predicted probability with the actual likelihood of correctness. A perfectly calibrated model will output a 70% confidence score that is correct exactly 70% of the time. Expected Calibration Error (ECE) is the standard metric, measuring the weighted average gap between confidence and accuracy across bins. Temperature scaling is the most common and effective post-hoc calibration method, as it preserves accuracy while dramatically reducing ECE.

A parametric calibration method that fits a logistic regression model to the raw output scores of a binary classifier. Unlike temperature scaling, which uses a single scalar parameter, Platt scaling learns two parameters: a slope and an intercept. This makes it slightly more flexible but also more prone to overfitting on small calibration datasets. It is the precursor to temperature scaling and remains useful when the calibration set is large enough to support two-parameter optimization.

A non-parametric calibration method that learns a piecewise constant, monotonically increasing function to map model scores to calibrated probabilities. It is more flexible than temperature or Platt scaling but requires significantly more calibration data to avoid overfitting. Isotonic regression can correct any monotonic distortion in the probability curve, making it powerful when the model's uncalibrated probabilities are systematically skewed in complex ways.

The primary metric for evaluating calibration quality. ECE partitions predictions into M equally-spaced confidence bins and computes the weighted average of the absolute difference between each bin's accuracy and its average confidence. A lower ECE indicates better calibration. Reliability diagrams visually plot bin accuracy against confidence, where perfect calibration follows the identity line. Temperature scaling typically reduces ECE by an order of magnitude on modern neural networks.

The model's uncertainty arising from a lack of knowledge or data, which is reducible with more training samples. Temperature scaling primarily addresses aleatoric uncertainty (inherent data noise) by calibrating the SoftMax output, but it does not directly model epistemic uncertainty. For open-set recognition, epistemic uncertainty is the critical signal—a high value indicates the input is far from the training distribution. Methods like Deep Ensembles and Bayesian neural networks explicitly model this uncertainty.