Factual Consistency

A verification methodology that breaks a generated summary into minimal, self-contained factual claims. Each atomic fact is individually verified against the source text using Natural Language Inference (NLI) models.

• Process: Summary → split into discrete claims → each claim checked for entailment
• Granularity: A single subject-predicate-object triple per fact
• Advantage: Pinpoints exactly which part of a summary is hallucinated
• Example: 'The defendant breached the contract on March 3' decomposes into: (1) a defendant exists, (2) a contract exists, (3) breach occurred, (4) date is March 3

NLI models classify the relationship between a premise (source document) and a hypothesis (summary claim) into three categories: entailment, contradiction, or neutral. This provides a structured framework for automated faithfulness evaluation.

• Entailment: The source text logically supports the claim
• Contradiction: The source text directly refutes the claim
• Neutral: The source text provides insufficient information
• Legal adaptation: Fine-tuned on legal NLI datasets like Law-NLI for domain accuracy

The technique of explicitly linking each factual statement in a generated summary back to its precise location in the source document. This creates an auditable chain of evidence.

• Implementation: Span-level citations with document ID, page, and paragraph references
• Format: 'The plaintiff filed on June 1 [Doc_A, ¶ 14]'
• Verification: Automated cross-checking against a citation verification system
• Legal necessity: Enables attorneys to rapidly validate AI outputs against primary sources

A metric quantifying the frequency at which a language model generates factually incorrect or unverifiable information not grounded in the source text. Critical for legal applications where fabrication carries professional liability.

• Calculation: (Number of hallucinated facts / Total atomic facts) × 100
• Types measured: Entity errors, date fabrication, relationship invention, numerical distortion
• Target threshold: Legal-grade systems aim for <1% hallucination rate
• Monitoring: Continuous evaluation in production using automated fact-checking pipelines

Ensuring that entities referred to by different expressions (pronouns, aliases, titles) are correctly tracked and consistently represented throughout the summary. Failure here causes entity confusion.

• Challenge: 'He' and 'the CEO' and 'John Smith' must resolve to the same entity
• Legal complexity: Multiple parties with similar roles across documents
• Solution: Coreference resolution preprocessing before summarization
• Validation: Post-hoc entity linkage verification against source document clusters

The accurate retention of chronological relationships, deadlines, and effective dates from source documents. Temporal distortion is a common failure mode in legal summarization.

• Critical elements: Dates, durations, sequences ('before', 'after', 'within 30 days')
• Failure example: Summarizing a sequence of events in reverse order
• Mitigation: Temporal relation extraction models that tag and align timeline events
• Validation: Timeline reconstruction from summary and comparison to source timeline

A quantitative metric measuring the frequency at which a language model generates factually incorrect or unverifiable information not grounded in the source text.

• Calculated as the ratio of fabricated claims to total generated statements
• In legal contexts, even a 1% hallucination rate can be catastrophic for case outcomes
• Often measured through human annotation or automated NLI-based detectors
• Directly inversely correlated with factual consistency scores

A task where a model determines whether a hypothesis (summary statement) is entailed by, contradicts, or is neutral to a given premise (source text).

• Serves as an automated factual consistency verifier
• Models like DeBERTa fine-tuned on NLI datasets can flag unsupported claims
• Enables scalable evaluation without requiring human reviewers for every output
• Forms the backbone of summary faithfulness scoring pipelines

A method for evaluating summary faithfulness by breaking down generated text into minimal, self-contained factual claims for individual verification against the source.

• Each atomic fact represents a single verifiable proposition
• Enables granular auditing of which specific claims are supported or hallucinated
• Used in frameworks like FActScore to compute precision at the fact level
• Particularly valuable in legal contexts where a single misstated fact can alter case interpretation

The technique of explicitly linking each factual statement in a generated summary back to its precise location in the source document.

• Provides verifiable provenance for every claim
• Often implemented through span-level citation markers or footnote-style references
• Enables rapid human verification by directing reviewers to exact passages
• Critical for legal workflows where citation integrity is non-negotiable

The NLP task of identifying all linguistic expressions that refer to the same real-world entity across a document.

• Resolves pronouns, aliases, and definite descriptions to canonical entities
• Essential for merging facts about a specific party, contract, or statute in legal texts
• Errors in coreference directly cause factual inconsistency by attributing actions to wrong entities
• Modern systems use span-based neural architectures for high accuracy on legal prose

The task of identifying and linking semantically related passages or entities that discuss the same event or fact across a collection of distinct documents.

• Enables multi-document consistency verification
• Detects contradictions between witness statements, contract versions, or judicial opinions
• Uses embedding similarity and entity linking to establish cross-document connections
• Foundational for multi-document fusion without introducing factual conflicts