Entailment Scoring

The foundational NLP task that powers entailment scoring. An NLI model classifies the logical relationship between a premise (evidence text) and a hypothesis (claim) into one of three categories:

• Entailment: The premise logically implies the hypothesis.
• Contradiction: The premise negates the hypothesis.
• Neutral: The premise neither confirms nor denies the hypothesis.

Modern NLI models like RoBERTa-MNLI are fine-tuned on large annotated datasets and serve as the computational engine for scoring claim-evidence alignment.

An evaluation score measuring whether a generated text is factually consistent with its source document. Unlike simple entailment, faithfulness metrics specifically detect extrinsic hallucinations—claims that cannot be directly inferred from the provided context.

Key approaches include:

• NLI-based scoring: Decomposing generated text into atomic claims and checking each against the source via entailment.
• Question-Answering based: Generating questions from the output and verifying if the source can answer them.

A score of 1.0 indicates perfect alignment; lower scores flag potential fabrications.

The automated identification of logically incompatible statements. In verification pipelines, contradiction detection serves as a binary filter—if a claim contradicts established evidence, it is immediately flagged as false regardless of other signals.

Common architectures:

• Sentence-pair models: Process claim and evidence jointly through cross-attention.
• Decompositional approaches: Break complex claims into atomic propositions and check each independently.

Contradiction detection is critical for catching intra-document inconsistencies and conflicts between generated text and retrieved evidence.

The task of isolating specific text spans from a source document that directly support or refute a claim. This is a prerequisite step for entailment scoring—you cannot score entailment without first identifying the relevant evidence.

Modern approaches include:

• Span prediction models: Fine-tuned extractive QA models that output start/end token positions.
• Sentence-level classifiers: Score each sentence independently for relevance to the claim.
• LLM-based extraction: Prompt large models to cite supporting passages verbatim.

High-quality evidence extraction directly improves downstream entailment accuracy by reducing noise.

The process of aligning a model's predicted probability with its actual likelihood of correctness. A well-calibrated entailment scorer should produce scores where 0.9 means 90% chance of true entailment, not merely a high relative ranking.

Calibration techniques:

• Temperature scaling: A post-hoc method that learns a single parameter to soften or sharpen output probabilities.
• Platt scaling: Fits a logistic regression model on top of raw logit scores.
• Isotonic regression: A non-parametric approach that learns a monotonic mapping from scores to empirical accuracy.

Poorly calibrated scores undermine trust in automated verification systems.

A hallucination reduction method where a model self-audits its output through iterative questioning. The process:

1. Draft: Generate an initial response.
2. Verify: Produce a series of independent fact-checking questions based on the draft.
3. Answer: Answer each verification question independently.
4. Correct: Revise the original response to align with verified answers.

CoVe uses entailment scoring implicitly—the final correction step resolves contradictions between the draft and verified facts, producing a more faithful output without external retrieval.