Glossary

FActScore

A fine-grained evaluation metric that decomposes generated text into atomic facts and verifies each independently against a trusted knowledge source to calculate a factual precision score.

Get in touch Learn more

Moody home-office setup in a converted highrise loft, analyst working late with multiple screens showing knowledge graph visualizations, city lights through large windows behind.

FACTUAL PRECISION METRIC

What is FActScore?

FActScore is a fine-grained evaluation metric that quantifies the factual precision of a generated text by decomposing it into atomic facts and independently verifying each one against a trusted knowledge source.

FActScore is an automated evaluation metric that measures the factual precision of text generated by language models, specifically designed for biography generation tasks. It works by breaking a generated passage into a series of atomic facts—short, self-contained assertions—and then verifying each fact independently against a corresponding Wikipedia article using a natural language inference model. The final score is the percentage of atomic facts that are supported by the knowledge source, providing a granular, interpretable measure of hallucination.

Introduced by researchers at the University of Washington and Meta AI, FActScore addresses the limitations of holistic human evaluation by enabling fine-grained, scalable factuality auditing. The metric relies on a retrieval-augmented verification pipeline where each atomic fact is paired with a relevant passage from a trusted corpus, and a Natural Language Inference (NLI) model classifies the fact as supported, unsupported, or irrelevant. This methodology makes FActScore a critical tool for factual grounding and confidence calibration in generative AI systems.

FACTUAL PRECISION METRIC

Key Features of FActScore

FActScore decomposes generated text into atomic facts and independently verifies each against a trusted knowledge source, providing a fine-grained factual precision score.

Atomic Fact Decomposition

FActScore breaks a generated biography into minimal, self-contained atomic facts—each a single, indivisible assertion. For example, 'John Smith, born in Boston, is a professor at MIT' becomes:

John Smith was born in Boston
John Smith is a professor
John Smith works at MIT This granularity enables precise, per-fact verification rather than holistic judgments.

Independent Knowledge Source Verification

Each atomic fact is verified against a trusted reference corpus, typically Wikipedia. The system uses a Natural Language Inference (NLI) model to classify each fact as Supported, Unsupported, or Irrelevant based on the retrieved evidence. This decouples verification from the generator's own knowledge, providing an objective accuracy measure.

Factual Precision Scoring

The core metric is calculated as the fraction of atomic facts verified as Supported by the knowledge source. A score of 0.85 means 85% of the generated claims were factually grounded. This provides a transparent, interpretable number that directly quantifies hallucination rates, unlike perplexity or BLEU scores which correlate poorly with factuality.

Human-Correlated Evaluation

FActScore was validated against human judgments of factuality, showing strong correlation with expert assessments. The metric identifies hallucinated entities, incorrect relations, and unsupported claims in a way that aligns with how human fact-checkers evaluate text, making it a reliable automated proxy for manual verification.

Model-Agnostic Benchmarking

FActScore evaluates any text generation system regardless of architecture. It has been used to benchmark models like GPT-4, Claude, and PaLM on biography generation tasks, revealing that even state-of-the-art models hallucinate 15-30% of atomic facts when generating content about less prominent entities.

Granular Error Analysis

Beyond a single score, FActScore enables detailed error categorization:

Entity errors: Wrong person, place, or organization
Relation errors: Incorrect attribute or association
Temporal errors: Wrong dates or time periods This breakdown helps developers target specific failure modes in their generation pipelines.

FACTSCORE EXPLAINED

Frequently Asked Questions

Explore the mechanics, applications, and limitations of FActScore, the fine-grained evaluation metric for measuring factual precision in AI-generated text.

FActScore is a fine-grained evaluation metric that quantifies the factual precision of a generated text by decomposing it into atomic facts and verifying each one independently against a trusted knowledge source. The process begins by breaking a generated passage, such as a biography, into a list of self-contained, indivisible factual claims. Each atomic fact is then checked against a specified knowledge corpus, typically Wikipedia, using a Natural Language Inference (NLI) model or a prompted large language model. The model determines if the fact is supported, refuted, or has insufficient evidence. The final FActScore is calculated as the percentage of supported atomic facts out of the total, providing a direct, interpretable measure of factual precision that correlates well with human judgments of truthfulness.

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

FACTUAL PRECISION COMPARISON

FActScore vs. Other Evaluation Metrics

A comparison of FActScore against other common metrics used to evaluate the factual accuracy and grounding of generated text.

Feature	FActScore	ROUGE	BLEU	Factual Consistency
Granularity	Atomic fact-level	N-gram overlap	N-gram precision	Sentence/span-level
Verification Method	External KB (Wikipedia)	Reference text overlap	Reference text overlap	Source document entailment
Measures Factual Precision
Measures Recall/Completeness
Requires Reference Text
Requires External Knowledge Base
Hallucination Detection Sensitivity	High	None	None	Medium
Human Correlation	0.71 Pearson r	0.25-0.40 Pearson r	0.20-0.35 Pearson r	0.50-0.65 Pearson r

FACTUAL GROUNDING ECOSYSTEM

Related Terms

FActScore relies on a broader ecosystem of verification, retrieval, and knowledge representation techniques. These related concepts form the technical foundation for fine-grained factual evaluation.

Atomic Fact

A minimal, self-contained, and indivisible piece of information expressed in a single sentence. FActScore's core methodology depends on decomposing generated text into atomic facts before verification.

Each atomic fact contains exactly one piece of verifiable information
Example: "Ada Lovelace was born in London" is atomic; "Ada Lovelace, a mathematician, was born in London in 1815" contains three atomic facts
Decomposition enables fine-grained precision scoring rather than holistic judgments

Natural Language Inference (NLI)

A foundational NLP task that determines whether a hypothesis can be logically inferred from a premise. FActScore uses NLI models as automated verifiers to check if a knowledge source entails each atomic fact.

Three possible labels: entailment, contradiction, or neutral
Modern NLI models like DeBERTa achieve over 90% accuracy on benchmark datasets
Serves as the computational engine for automated factual verification at scale

Entity Linking

The process of identifying textual mentions of named entities and disambiguating them by linking to unique canonical identifiers in a knowledge graph. Critical for FActScore because verification requires knowing exactly which entity is being discussed.

Links "Paris" in text to Q90 (capital of France) vs Q956 (mythological figure) in Wikidata
Prevents verification errors caused by entity ambiguity
Enables precise retrieval of the correct Wikipedia article for fact-checking

Retrieval-Augmented Generation (RAG)

A framework that grounds language model responses by first retrieving relevant information from an external knowledge base, then augmenting the prompt with this context before generation. FActScore evaluates the factual precision of RAG outputs by measuring how well retrieved evidence supports generated claims.

Combines dense retrieval with generative models
FActScore can serve as an evaluation metric for RAG system quality
High FActScore indicates strong retrieval-to-generation fidelity

Semantic Triples

A data structure consisting of a subject, predicate, and object that represents a single factual assertion. Each atomic fact in FActScore can be mapped to one or more semantic triples for structured verification against knowledge graphs.

Format: <Ada Lovelace> <bornIn> <London>
Forms the foundational unit of knowledge graphs and RDF stores
Enables programmatic fact-checking against structured databases like Wikidata

Chain-of-Verification (CoVe)

A prompting technique where a language model first drafts a response, then generates a series of independent fact-checking questions to systematically verify and correct its own output. Complements FActScore by providing a self-verification mechanism that can be evaluated externally.

Reduces hallucination through structured self-critique
Each verification question targets a specific atomic fact
FActScore can measure the improvement in factual precision after CoVe is applied

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.