Hallucination Entropy

When a model encounters a prompt outside its training distribution, the probability mass flattens across many possible tokens. This high entropy state indicates the model is guessing rather than retrieving. Monitoring the entropy of generated sequences in real-time allows systems to flag potential fabrications before they reach the end-user. A sudden spike in entropy mid-generation is a strong signal that the model has lost its factual grounding.

While a confidence score is a post-hoc probability assigned to a final output, hallucination entropy measures the uncertainty during the token selection process itself. A model can output a high-confidence score for a hallucinated fact if it is confidently wrong. Entropy analysis of the logit distribution provides a more honest, real-time view of the model's internal state, distinguishing between epistemic uncertainty (lack of knowledge) and aleatoric uncertainty (inherent data noise).

Entropy metrics can be used to dynamically adjust decoding strategies to mitigate hallucinations. Common techniques include:

• Threshold Gating: If token entropy exceeds a set threshold, the system can refuse to answer or fall back to a retrieval step.
• Adaptive Temperature Scaling: Dynamically lowering the temperature when entropy spikes forces the model toward more deterministic, lower-risk outputs.
• Nucleus Sampling (Top-p) Tuning: Tightening the top_p value restricts the candidate pool to only the most probable tokens when uncertainty is high.

The primary metric is the predictive entropy of the output probability distribution over the vocabulary at each time step. It is calculated as the negative sum of the log-probabilities of all possible tokens. A closely related metric is token surprisal, which is the negative log-probability of the specific token that was actually chosen. High surprisal indicates the model selected a low-probability token, a common signature of hallucination.

Standard token-level entropy can be noisy. Semantic entropy clusters generated sequences by their underlying meaning rather than exact token matches. If a model generates multiple high-probability paraphrases that mean the same thing, token entropy is high but semantic entropy is low—indicating a stable fact. If the model generates semantically distinct, contradictory answers with similar probabilities, semantic entropy is high, signaling a high risk of hallucination.

In Retrieval-Augmented Generation (RAG), hallucination entropy serves as a feedback signal. A high entropy score on a generated passage can trigger a secondary retrieval loop to fetch more specific documents. This creates a closed-loop system where the model's own uncertainty directly drives a corrective action, grounding the output in fresh, verifiable data from the vector database before finalizing the response.

A quantitative metric, often a probability or percentage, assigned by an AI model to indicate the likelihood that its generated output is factually correct and reliable. A high hallucination entropy state typically produces a miscalibrated confidence score—the model may express high confidence in a fabricated statement. Effective calibration ensures the score accurately reflects the true probability of correctness, serving as a primary signal for downstream filtering or human review.

The uncertainty in an AI model's prediction caused by a lack of knowledge or training data, which can theoretically be reduced with more information. This is a direct driver of hallucination entropy. When a model encounters a query outside its training distribution, epistemic uncertainty spikes, increasing the randomness of token generation. Unlike aleatoric uncertainty, this can be addressed by retrieving more data or expanding the training corpus.

A specific, highly effective logit calibration method that uses a single scalar parameter to soften or sharpen the output probability distribution of a model. By adjusting the temperature, engineers can directly control hallucination entropy. A higher temperature flattens the distribution, increasing randomness and creativity but also the risk of fabrication. A lower temperature sharpens the distribution toward deterministic, high-probability tokens, reducing entropy at the cost of diversity.

A primary metric for measuring model calibration by partitioning predictions into bins and computing the weighted average of the difference between accuracy and confidence in each bin. ECE provides a quantitative lens on hallucination entropy's downstream effects. A model with high entropy will show a large ECE, as its confidence scores fail to predict actual correctness. Monitoring ECE is essential for detecting calibration drift in production systems.

A model-agnostic framework that generates prediction sets with a mathematically guaranteed coverage probability, providing a rigorous alternative to single confidence scores. Instead of outputting one token with an unreliable probability, conformal prediction produces a set of plausible outputs. This directly addresses hallucination entropy by acknowledging uncertainty explicitly—if the entropy is too high, the prediction set will be large, signaling low confidence to the user or downstream system.

A metric evaluating how well an AI-generated statement is supported by verifiable evidence from a specific, retrieved knowledge source. This score serves as a counter-signal to hallucination entropy. Even if token-level entropy is high, a strong factual grounding score—derived from a retrieved document—can override the randomness. Conversely, low grounding combined with high entropy is a reliable predictor of hallucination, making this metric critical for RAG system guardrails.