Confidence Score

Confidence scores are typically derived from the softmax output of the final neural network layer. For each token generated, the model assigns a probability between 0 and 1. A score of 0.95 means the model estimates a 95% likelihood that the specific token is correct given the preceding context. These raw probabilities are often overconfident and require calibration to reflect true empirical likelihoods.

A single confidence score for a full generated response is computed by aggregating token-level probabilities. Common methods include:

• Mean token probability: The arithmetic average of all token scores.
• Geometric mean: Penalizes low-probability tokens more heavily.
• Minimum token probability: The score of the least confident token, useful for identifying potential hallucination points.
• Length-normalized log probability: A standard metric used in benchmarks like MMLU to compare sequences of different lengths.

A model can be accurate but miscalibrated. Calibration measures the alignment between a model's predicted probability and its actual empirical frequency of being correct. If a model says it is 90% confident across 100 predictions, a perfectly calibrated model will be correct exactly 90 times. Modern large language models often exhibit overconfidence, where their stated confidence exceeds their actual accuracy, requiring post-hoc recalibration techniques like Platt scaling or temperature scaling.

Before softmax normalization, the model outputs raw logits—unnormalized scores for each token in the vocabulary. The confidence score is the result of applying the softmax function to these logits. Analyzing logit distributions can reveal model uncertainty: a flat distribution across many tokens indicates high uncertainty, while a sharp peak on one token indicates high confidence. This raw signal is more informative than the final probability for detecting edge cases.

Low confidence scores serve as a primary signal for potential hallucination. In RAG systems, if a generated claim about a retrieved document has a confidence score below a defined threshold, the system can trigger corrective actions such as:

• Re-querying the knowledge base with a reformulated search.
• Refusing to answer and requesting clarification.
• Flagging the output for human review. This threshold-based gating is a foundational guardrail in production LLM deployments.

Beyond token probabilities, confidence can be assessed by analyzing the topology of the model's internal representations. Techniques like Mahalanobis distance measure how far a test input's embedding is from the training distribution's class centroids. A large distance indicates the input is out-of-distribution, and the model's softmax confidence is likely unreliable. This method provides a complementary uncertainty signal that does not rely on the output layer's probabilities.

The process of aligning a model's predicted confidence score with its actual empirical accuracy. A perfectly calibrated model that outputs 80% confidence will be correct exactly 80% of the time. Poor calibration leads to overconfident errors or underconfident correct answers. Measured using Expected Calibration Error (ECE) and visualized with reliability diagrams. Techniques like temperature scaling and Platt scaling are applied post-training to recalibrate raw logit outputs.

An evaluation score measuring whether a generated text is factually consistent with and directly inferable from the provided source context. Unlike confidence, which is the model's internal belief, faithfulness is an external verification against a ground-truth document. Common implementations use Natural Language Inference (NLI) models to detect contradictions. A high-confidence but unfaithful output indicates a hallucination where the model believes its own fabrication.

The frequency at which a model generates factually incorrect or nonsensical content unsupported by training data or context. Confidence scores serve as a first-line filter: outputs below a threshold can be suppressed or flagged. However, high confidence does not guarantee non-hallucination. Advanced detection pairs confidence with entailment scoring and retrieval augmentation to cross-reference claims against trusted knowledge bases.

Uses a Natural Language Inference model to calculate the probability that a given evidence text logically implies a target claim. Outputs are classified as entailment, contradiction, or neutral. This provides an objective, source-grounded alternative to raw model confidence. In RAG pipelines, entailment scores verify whether the generated answer is actually supported by the retrieved document, creating a factual consistency check independent of the generator's self-assessment.

A guardrail mechanism using a model's perplexity—the inverse probability of a sequence normalized by length—to detect low-quality outputs. High perplexity indicates the model found the text surprising or improbable, often correlating with hallucinations or nonsensical generation. Unlike confidence scores derived from single-token probabilities, perplexity evaluates the fluency and coherence of the entire sequence, making it a complementary signal for output filtering.

A hallucination reduction method where a model drafts a response, then generates independent verification questions to fact-check its own output. Each question is answered against retrieved evidence, and the final response is revised based on verification results. This creates a self-critique loop that decouples confidence from correctness. The model's initial confidence may be high, but CoVe systematically identifies and corrects unsupported claims before the final output.