Confidence Calibration

The defining characteristic of a calibrated model is the identity function between predicted confidence and observed accuracy. When a model assigns a 70% probability to a set of predictions, exactly 70% of those predictions should be correct. This is measured using Reliability Diagrams, which plot expected confidence against actual accuracy. A perfectly calibrated model produces a diagonal line. Deviations indicate overconfidence (confidence > accuracy) or underconfidence (confidence < accuracy).

A calibrated model distinguishes between aleatoric uncertainty (inherent noise in the data, irreducible) and epistemic uncertainty (uncertainty due to lack of knowledge, reducible with more data). A well-calibrated model expresses high epistemic uncertainty on out-of-distribution inputs by producing a wide, low-confidence predictive distribution rather than a falsely confident, peaked one. This is critical for rejection classification, where the model abstains from deciding on ambiguous cases.

The primary diagnostic metric is Expected Calibration Error (ECE). ECE works by:

• Binning all predictions by their confidence score (e.g., 0-10%, 10-20%).
• Calculating the difference between average confidence and actual accuracy within each bin.
• Computing a weighted average of these gaps. A lower ECE indicates better calibration. However, ECE suffers from binning bias, so advanced metrics like Adaptive Calibration Error (ACE) and Brier Score are used for more granular evaluation.

Modern neural networks are notoriously miscalibrated, often exhibiting extreme overconfidence. Temperature Scaling is the most effective post-hoc method to fix this. It introduces a single scalar parameter, T (temperature), that softens the softmax output distribution without altering the model's accuracy or rank order. When T > 1, the output probabilities become more diffuse and better aligned with empirical likelihoods. This parameter is optimized on a held-out validation set using Negative Log Likelihood (NLL).

A truly calibrated model maintains its probability alignment even when the input data distribution changes. A model calibrated on clean, in-distribution test data often becomes severely overconfident on corrupted or shifted data. Techniques like mixup training and stochastic weight averaging (SWA) improve calibration robustness by implicitly training the model on a wider, smoother manifold, preventing it from assigning 100% confidence to spurious, brittle features.

Calibration is a prerequisite for more rigorous uncertainty quantification frameworks like Conformal Prediction. While calibration aligns probabilities, conformal prediction wraps a calibrated model to produce statistically valid prediction sets with a guaranteed marginal coverage probability. For a user-specified significance level α (e.g., 0.1), a conformal predictor guarantees the true label is in the set with probability 1-α, assuming exchangeable data. This transforms a heuristic confidence score into a rigorous, frequentist guarantee.

A model-agnostic, distribution-free framework that wraps any predictive model to produce statistically rigorous prediction sets with a guaranteed coverage probability. Instead of a single point estimate, it outputs a set of plausible labels (e.g., 'cat, dog, fox') with a mathematical guarantee that the true label is included, say, 95% of the time. This directly operationalizes confidence calibration by providing finite-sample validity without requiring strong distributional assumptions.

A measure of uncertainty that clusters token-level predictions by their semantic meaning before calculating entropy. Unlike naive entropy, which treats 'Paris' and 'the capital of France' as distinct uncertainties, semantic entropy recognizes them as equivalent answers. This distinguishes between lexical variation (high token entropy, low semantic entropy) and genuine factual indecision (high semantic entropy), making it a powerful signal for detecting hallucinations.

A metric quantifying the uncertainty or randomness in a language model's output distribution, used as a predictive signal for detecting confabulated or non-factual text. When a model's internal probability mass is spread thinly across many unrelated tokens, hallucination entropy spikes. This serves as a real-time, white-box calibration check, flagging outputs where the model's confidence is poorly calibrated against reality.

An automated evaluation process that measures the degree to which a generated summary or statement aligns with the facts presented in a source document. It penalizes contradictions and hallucinations by comparing atomic claims against the ground truth text. This provides an empirical accuracy check against which a model's self-reported confidence score can be compared, closing the calibration loop.

A prompting technique where a language model first drafts a response, then generates a series of independent fact-checking questions to systematically verify and correct its own initial output. By decoupling generation from verification, CoVe reduces the compounding of errors and provides a structured audit trail. The model's ability to self-correct is a direct test of its internal confidence calibration.

A metric evaluating how accurately a generated statement's citations point to the specific source passages that directly support it. High attribution fidelity means a reference is not just topically relevant but precisely evidential. This is the external, verifiable counterpart to internal confidence calibration—it proves that a model's high-confidence claim is grounded in a retrievable, authoritative source.