Inferensys

Glossary

Atomic Fact Decomposition

A method for evaluating summary faithfulness by breaking down a generated text into minimal, self-contained factual claims for individual verification against the source.
Legal team reviewing EU AI Act compliance documents on laptop in modern office, coffee cups and papers on table, casual meeting.
SUMMARY FAITHFULNESS EVALUATION

What is Atomic Fact Decomposition?

A granular evaluation method that breaks generated text into minimal, self-contained factual claims for individual verification against a source document.

Atomic Fact Decomposition is a method for evaluating summary faithfulness by segmenting a generated text into a list of minimal, self-contained factual assertions—each representing a single, verifiable piece of information. These indivisible 'atomic facts' are then individually checked against the source document to determine if they are supported, contradicted, or unverifiable, providing a fine-grained precision metric for hallucination detection.

This technique moves beyond surface-level n-gram overlap metrics like ROUGE by assessing factual consistency at the semantic level. By requiring each atomic unit to be verified, the method creates a rigorous audit trail for source attribution, enabling developers to precisely quantify a model's hallucination rate and identify specific failure modes in legal abstractive summarization systems where factual integrity is paramount.

DECOMPOSITION MECHANICS

Key Characteristics

The core operational principles that define how a generated summary is systematically dismantled into verifiable, atomic units of truth.

01

Minimal Self-Contained Units

Each decomposed fact must be a standalone assertion that can be verified as true or false without requiring context from other sentences. This prevents dependency chains where one error cascades.

  • Single Predicate Rule: One subject-predicate-object relationship per atom
  • No Anaphora: Pronouns and implicit references are resolved to explicit named entities
  • Example: 'The court ruled on March 3, 2024' is atomic; 'It was decided then' is not
02

Binary Verification Protocol

Every atomic fact is individually checked against the source document using a Natural Language Inference (NLI) model. The result is a strict binary classification with no partial credit.

  • Entailment: The source text logically supports the fact
  • Contradiction: The source text directly refutes the fact
  • Neutral: The source text does not mention or confirm the fact (treated as a hallucination)
  • Hallucination Rate Formula: Contradictions + Neutrals / Total Atoms
03

Granularity Trade-Off

The level of decomposition directly impacts evaluation precision. Over-decomposition creates noise; under-decomposition masks errors within complex statements.

  • Coarse-Grained: Sentence-level decomposition risks accepting compound falsehoods
  • Fine-Grained: Clause-level decomposition catches subtle misattributions
  • Optimal Balance: Decompose to the smallest unit that still expresses a complete proposition
  • Legal Example: 'The plaintiff, who filed on Tuesday, sought damages' becomes two atoms: (1) plaintiff filed Tuesday, (2) plaintiff sought damages
04

Entity Grounding

All entities referenced in atomic facts must be resolved to their canonical forms in the source text. This prevents evaluation failures caused by surface-form variation.

  • Coreference Resolution: 'The company,' 'it,' and 'Acme Corp' are normalized to a single entity ID
  • Date Normalization: 'Last Tuesday' becomes an absolute date based on document metadata
  • Named Entity Linking: Parties, judges, and statutes are linked to a knowledge base for cross-document consistency
05

Source Attribution Mapping

Each atomic fact retains a pointer to its precise origin in the generated summary and the source document. This creates a full audit trail for every claim.

  • Summary Span: Character-level index of where the fact was expressed in the summary
  • Source Span: Paragraph or line citation in the original document used for verification
  • Confidence Score: The NLI model's probability score for the entailment classification
  • Utility: Enables targeted human review of only the facts flagged as unsupported
06

Completeness vs. Conciseness

Atomic decomposition evaluates faithfulness, not coverage. A summary can be perfectly faithful (all atoms entailed) yet incomplete (missing key source facts).

  • Faithfulness: All stated facts are true (measured by atomic decomposition)
  • Completeness: All important source facts are included (measured by recall-oriented metrics)
  • Trade-off: Increasing completeness often introduces hallucination risk
  • Combined Metric: F1 score balancing ROUGE recall against atomic fact precision
ATOMIC FACT DECOMPOSITION

Frequently Asked Questions

Explore the core concepts behind evaluating summary faithfulness by breaking down generated text into minimal, verifiable claims. These answers target the most common technical queries about this critical evaluation methodology.

Atomic fact decomposition is a faithfulness evaluation method that breaks a generated summary into a list of minimal, self-contained factual claims—each representing a single, indivisible piece of information. The process works by parsing the summary text and segmenting it into discrete propositions that can be independently verified against the source document. For example, the sentence "The court ruled on Tuesday that the patent was invalid due to prior art" decomposes into three atoms: (1) a ruling occurred, (2) the ruling date was Tuesday, and (3) the patent was invalidated because of prior art. Each atom is then checked against the source using a Natural Language Inference (NLI) model, which classifies it as "entailed," "contradicted," or "neutral." The final hallucination rate is calculated as the proportion of atoms that are not directly supported by the source text, providing a granular, interpretable metric for summary quality.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.