Grounded BERTScore

Unlike standard BERTScore which compares generated text to a reference, Grounded BERTScore computes cosine similarity between each generated token and its most similar token in the source evidence document. This alignment identifies which generated tokens have contextual support and which are hallucinated.

• Uses contextual embeddings from models like BERT or RoBERTa
• Computes pairwise similarity between generated and source token representations
• Produces a token-level grounding map showing supported vs. unsupported spans

Grounded BERTScore decomposes factual grounding into three interpretable metrics:

• Grounded Precision: The fraction of generated tokens that have a sufficiently similar match in the source evidence, penalizing hallucinated content
• Grounded Recall: The fraction of source evidence tokens captured in the generated text, measuring completeness of evidence usage
• Grounded F1: The harmonic mean of precision and recall, providing a single balanced grounding score

A low precision score directly indicates unsupported generation, while low recall signals incomplete evidence coverage.

A critical hyperparameter in Grounded BERTScore is the similarity threshold that determines whether a token pair is considered grounded. Tokens with maximum cosine similarity below this threshold are flagged as unsupported.

• Typical thresholds range from 0.5 to 0.7 depending on domain specificity
• Higher thresholds enforce stricter grounding requirements
• Threshold tuning allows adaptation to different factual precision needs, from creative summarization to strict regulatory compliance

Grounded BERTScore enables fine-grained hallucination detection by identifying exactly which tokens in a generated sequence lack evidential support. This token-level attribution is more actionable than sentence-level or passage-level hallucination flags.

• Produces a binary grounding mask over the generated sequence
• Enables precise localization of factual errors for downstream correction
• Integrates with attribution-aware chunking pipelines to map unsupported tokens back to source gaps

Grounded BERTScore serves as a core faithfulness metric in RAG evaluation frameworks, complementing retrieval metrics like NDCG and generation metrics like BLEU. It specifically answers: 'Is the generated text actually supported by what was retrieved?'

• Used alongside Natural Language Inference (NLI) models for multi-faceted factual verification
• Provides a continuous score suitable for automated regression testing in CI/CD pipelines
• Enables systematic comparison of different retrieval and generation configurations on factual consistency

Grounded BERTScore has known failure modes that practitioners must account for:

• Semantic drift: Tokens may have high cosine similarity but different meanings in context, producing false positives
• Paraphrase penalty: Legitimate paraphrasing of source material may fall below the similarity threshold, producing false negatives
• Embedding model sensitivity: Scores vary significantly based on the underlying embedding model's domain training
• Does not detect logical contradictions where generated text is semantically similar to source but asserts opposite claims

A quantitative evaluation score measuring the degree to which a generated statement is logically entailed by the provided source context. Unlike Grounded BERTScore, which computes token-level semantic similarity, faithfulness metrics often leverage Natural Language Inference (NLI) models to detect contradictions.

• Entailment: The hypothesis is fully supported by the premise.
• Contradiction: The hypothesis is refuted by the premise.
• Neutral: The hypothesis cannot be determined from the premise.

This metric is critical for hallucination detection in summarization and RAG pipelines, ensuring outputs don't introduce unsupported facts.

The process of identifying and linking specific spans of generated text to the exact source documents or data records that support them. While Grounded BERTScore evaluates if a statement is supported, citation attribution pinpoints where the support originates.

• Inline Citation: Direct reference markers inserted into the text span.
• Post-hoc Attribution: Linking claims to sources after generation.
• Attribution-Aware Chunking: Preserving source metadata during document preprocessing.

This mechanism enables verifiable output, allowing users and auditors to trace every claim back to its origin.

A constrained text generation strategy that manipulates token probabilities during inference to favor words and phrases explicitly supported by a provided evidence document. Unlike Grounded BERTScore, which is a post-hoc evaluation metric, grounded decoding is a proactive intervention during the generation process.

• Logit Manipulation: Boosting probabilities of tokens present in the source context.
• Constrained Beam Search: Pruning generation paths that diverge from evidence.
• Context-Aware Sampling: Penalizing hallucinations at the token level.

This technique directly reduces factual errors before they appear in the output.

An automated evaluation step that compares a generated summary or answer against its source material to identify contradictions, hallucinations, and unsupported inferences. Grounded BERTScore serves as one component within a broader consistency checking pipeline.

• Atomic Fact Decomposition: Breaking text into individual verifiable claims.
• Cross-Source Verification: Requiring multiple independent documents to corroborate a fact.
• Temporal Grounding: Anchoring information to a specific time range to prevent outdated facts.

This multi-layered approach ensures robust hallucination mitigation beyond simple semantic similarity.

A foundational NLP task where a model determines the directional logical relationship between a premise (source text) and a hypothesis (generated claim). NLI models are often used as the backbone for faithfulness metrics and groundedness checks.

• Entailment: The premise logically implies the hypothesis.
• Contradiction: The premise logically negates the hypothesis.
• Neutral: No logical relationship exists.

Grounded BERTScore differs by computing continuous semantic similarity rather than discrete logical relationships, making it more sensitive to partial support but less interpretable for strict fact-checking.

The process of validating generated factual statements by querying a structured knowledge graph to confirm the existence and correctness of subject-predicate-object triples. This provides deterministic, symbolic verification that complements the soft, distributional similarity measured by Grounded BERTScore.

• Entity Disambiguation: Resolving textual mentions to unique knowledge base identities.
• Triple Verification: Checking if a claimed relationship exists in the graph.
• Ontology Alignment: Mapping generated concepts to formal graph schemas.

This hybrid approach combines neural semantic matching with symbolic precision for high-stakes applications.