Semantic Entropy

Groups token-level predictions by semantic equivalence rather than surface form. For example, 'The capital of France is Paris' and 'Paris is the capital of France' are clustered together despite different token sequences. This clustering uses bidirectional entailment between generated sequences—two outputs belong to the same semantic cluster if they entail each other. The technique leverages natural language inference (NLI) models or LLM-based entailment checks to determine equivalence, transforming raw output distributions into a smaller set of meaning-distinct answer categories.

Computes uncertainty over semantic clusters rather than individual tokens. The formula applies Shannon entropy to the probability mass of each meaning cluster: H_semantic = -Σ p(c) log p(c), where p(c) is the sum of probabilities of all token sequences assigned to cluster c. This distinguishes between:

• High semantic entropy: Model is genuinely uncertain between multiple plausible answers (e.g., different historical interpretations)
• Low semantic entropy: Model is confident in meaning but varies in phrasing (e.g., paraphrasing the same fact)

Semantic entropy serves as a strong predictor of confabulation. Research shows that when a model hallucinates, it often generates outputs with high semantic entropy—the model lacks a consistent underlying belief and produces semantically diverse, contradictory completions. By thresholding semantic entropy, systems can flag potentially unreliable generations for human review or trigger fallback mechanisms. This approach outperforms naive token-level uncertainty metrics, which conflate lexical diversity with factual uncertainty.

Implementation requires multi-sample generation from the same prompt:

• Sample N diverse outputs using temperature sampling (typically N=5-10)
• Cluster outputs by semantic equivalence using entailment or embedding similarity
• Compute entropy over the resulting cluster distribution This Monte Carlo approximation of semantic uncertainty adds computational overhead but provides a principled decomposition unavailable from single-pass generation. The technique is model-agnostic and requires no access to internal weights.

Semantic entropy approximates epistemic uncertainty—the uncertainty arising from lack of knowledge—while filtering out aleatoric uncertainty from lexical variation. This connects to Bayesian deep learning concepts: semantic entropy estimates uncertainty over model meaning rather than model outputs. In principle, a perfectly calibrated model would exhibit low semantic entropy when it possesses the relevant knowledge and high semantic entropy when it does not, making it a useful signal for model self-assessment and selective prediction.

The foundational concept from which semantic entropy is derived. Standard predictive entropy measures the spread of a probability distribution over raw tokens. High entropy can indicate either data uncertainty (inherent ambiguity in the task) or model uncertainty (lack of knowledge). Semantic entropy refines this by first clustering token-level predictions by their semantic equivalence, then computing entropy over the resulting meaning clusters. This decomposition isolates uncertainty arising from multiple plausible answers rather than multiple plausible phrasings.

The algorithmic core of semantic entropy computation. After sampling multiple output sequences from a language model, each generation is grouped based on bidirectional entailment—two statements are in the same cluster if they logically entail one another. This is typically performed using a natural language inference model fine-tuned on datasets like MNLI. The result is a set of meaning-equivalent clusters, where variation within a cluster represents lexical diversity and variation across clusters represents genuine semantic uncertainty.

The standard Shannon entropy computed over a model's next-token probability distribution. For a given context, it is calculated as H = -Σ p(t_i) log p(t_i) across all tokens in the vocabulary. While useful as a broad uncertainty signal, it conflates aleatoric uncertainty from irreducible task ambiguity with epistemic uncertainty from the model's limited knowledge. Semantic entropy serves as a direct upgrade, filtering out the noise of synonymous token choices that inflate this raw metric.

The task of determining whether a hypothesis is true, false, or undetermined given a premise. In the semantic entropy pipeline, an NLI model acts as the arbiter of meaning equivalence. For two generated answers A and B, the system checks if A entails B and B entails A. This bidirectional entailment check is the gold standard for semantic clustering, as it captures logical equivalence even when surface forms differ dramatically. Models like DeBERTa fine-tuned on MNLI are commonly used for this role.

A complementary uncertainty quantification framework that provides distribution-free statistical guarantees. Unlike semantic entropy, which measures internal model uncertainty, conformal prediction produces prediction sets with a formal coverage guarantee—e.g., 'the true answer is in this set with 95% probability.' When combined with semantic entropy, conformal methods can calibrate the threshold at which a model should abstain from answering, turning a continuous uncertainty score into a decision rule with provable error control.

A decoding strategy that samples multiple reasoning paths and selects the most frequent final answer. It operates on the principle that a correct answer will emerge consistently across diverse reasoning traces, while errors will be scattered. Semantic entropy extends this idea by analyzing the distribution of meanings across samples rather than just the majority vote. A high self-consistency score with low semantic entropy indicates a confident, correct model; high self-consistency with high semantic entropy may signal a confidently wrong model.