Natural Language Inference (NLI)

NLI is formally a three-class classification problem. Given a premise P and a hypothesis H, a model must assign exactly one label:

• Entailment: H is definitely true given P. (P: 'A dog sleeps on a mat.' H: 'An animal is resting.')
• Contradiction: H is definitely false given P. (P: 'A dog sleeps on a mat.' H: 'The dog is running.')
• Neutral: H could be true or false; P does not provide sufficient information. (P: 'A dog sleeps on a mat.' H: 'The dog is a poodle.') This strict taxonomy forces models to learn logical consequence rather than mere topical similarity.

While often used interchangeably, a key distinction exists:

• Textual Entailment Recognition (RTE) is the original, broader task from the PASCAL challenges, often framed as a binary classification (Entailment vs. Non-Entailment).
• Natural Language Inference (NLI) is the modern, more rigorous formulation, standardizing the three-way classification (Entailment, Neutral, Contradiction) popularized by the SNLI and MultiNLI benchmarks. NLI's three-way split forces a model to distinguish between lack of evidence (Neutral) and contrary evidence (Contradiction), a critical capability for factual grounding.

NLI model development is driven by large-scale, crowd-sourced datasets that expose specific linguistic phenomena:

• SNLI (Stanford NLI): 570k sentence pairs grounded in image captions, establishing the modern three-way paradigm.
• MultiNLI: Extends SNLI across diverse genres (fiction, government reports, telephone speech), testing domain transfer.
• ANLI (Adversarial NLI): A human-and-model-in-the-loop dataset designed to be deliberately difficult, exposing brittle syntactic heuristics.
• WANLI: Uses a generator-evaluator loop to create high-quality, challenging examples that reduce artifacts found in crowd-sourced data. Performance on these benchmarks is the standard measure of a model's logical reasoning capability.

Two dominant neural architectures define NLI modeling:

• Siamese (Bi-Encoder): The premise and hypothesis are encoded independently by a shared transformer. The resulting [CLS] vectors are concatenated and fed to a classifier. This is computationally fast but limits token-level interaction.
• Cross-Encoder: The premise and hypothesis are concatenated with a [SEP] token and fed through a single transformer simultaneously. Full cross-attention between all tokens allows the model to learn subtle logical relationships, achieving higher accuracy at the cost of inference speed. Modern state-of-the-art systems typically use cross-encoders for accuracy and bi-encoders for efficient retrieval.

A critical finding in NLI research is that models often exploit annotation artifacts rather than learning true inference. Common shallow heuristics include:

• Lexical Overlap Hypothesis: If H contains many words from P, predict Entailment.
• Negation Heuristic: If H contains negation words like 'no' or 'never', predict Contradiction.
• Length Heuristic: If H is significantly longer than P, predict Neutral. These shortcuts cause brittle models that fail on adversarial test sets like ANLI. Debiasing techniques and adversarial training are essential to force models to learn genuine logical reasoning.

Beyond its standalone task, NLI models serve as powerful zero-shot classifiers for other NLP tasks without any task-specific training data. This is achieved by reformulating a target task as an entailment problem:

• Sentiment Analysis: P: 'The movie was a masterpiece.' H: 'The review is positive.' (Entailment)
• Topic Classification: P: 'The CPU clock speed is 4.2 GHz.' H: 'This text is about technology.' (Entailment)
• Fact Verification: P: 'The sky is blue.' H: 'The sky is blue.' (Entailment) This capability makes NLI a foundational task-agnostic reasoning primitive for building flexible AI systems.

An automated evaluation process that measures the degree to which a generated summary or statement aligns with the facts presented in a source document. It directly applies the entailment and contradiction signals from NLI models to penalize hallucinations. A high consistency score indicates that a hypothesis (the summary) is logically entailed by the premise (the source text), making it a critical guardrail in Retrieval-Augmented Generation (RAG) pipelines.

A measure of uncertainty in a language model's output that clusters token-level predictions by their semantic meaning before calculating entropy. Unlike naive entropy, which treats synonyms as distinct uncertainty, semantic entropy leverages NLI-based bidirectional entailment to group equivalent meanings. This allows the system to distinguish between benign lexical variation and genuine factual indecision, providing a more accurate signal for hallucination detection.

A prompting technique where a language model first drafts a response, then generates a series of independent fact-checking questions to systematically verify its own initial output. Each verification step implicitly performs an NLI task, checking if the original claim is entailed by the evidence retrieved from a trusted source. This self-correcting loop reduces hallucination rates without requiring external human feedback or complex model fine-tuning.

A minimal, self-contained, and indivisible piece of information expressed in a single sentence. Decomposing complex text into atomic facts is a prerequisite for fine-grained NLI. Each atomic fact serves as a distinct hypothesis that can be independently verified against a premise knowledge base. This decomposition is the foundational step for metrics like FActScore, enabling precise, granular factual verification.

A fine-grained evaluation metric that decomposes a generated biography into atomic facts and verifies each one independently against a trusted knowledge source like Wikipedia. The verification step is a direct application of NLI, classifying each atomic fact as supported (entailment) or unsupported (neutral/contradiction). The final score is the precision of supported facts, offering a highly interpretable measure of factual grounding.

A metric evaluating how accurately a generated statement's citations point to the specific source passages that directly support it. It goes beyond simple NLI entailment to measure citation precision. A statement might be factually true (entailed by some document), but attribution fidelity checks if the provided citation is the specific source that entails the claim, ensuring references are not just relevant but precisely evidential.