FaithDial

Every response in FaithDial is explicitly anchored to a knowledge source (Wikipedia snippet). The dataset was created by annotators who corrected hallucinated Wizard of Wikipedia responses, ensuring that all factual claims are entailed by the provided evidence. This makes FaithDial a benchmark for training models that resist the tendency to fabricate plausible-sounding but unsupported information.

• Annotators marked unsupported spans as hallucinations
• Corrections maintain conversational flow while ensuring strict entailment
• Enables training of hallucination-free dialogue models

FaithDial introduces a fine-grained classification system for factual errors, distinguishing between entity-level hallucinations (inventing people or places), relation-level hallucinations (fabricating connections between real entities), and sentence-level contradictions (statements that directly oppose the source). This taxonomy enables granular risk analysis and targeted model improvement.

• Entity errors: Wrong or invented named entities
• Relation errors: Incorrect associations between entities
• Contradiction errors: Statements that negate the knowledge source

The dataset was constructed using a rigorous Natural Language Inference (NLI) framework. Annotators were instructed to ensure that every corrected response could be logically deduced from the provided Wikipedia knowledge. This protocol transforms hallucination correction from a subjective task into a verifiable entailment verification process.

• Uses NLI principles for objective correction
• Each response must pass a logical deduction test
• Eliminates annotator subjectivity in defining faithfulness

FaithDial serves as both a training corpus and an evaluation benchmark for dialogue systems. Models fine-tuned on FaithDial demonstrate significantly lower hallucination rates while maintaining conversational quality. The dataset is commonly used alongside metrics like FActScore and Attribution Score to measure factual consistency.

• Used to fine-tune models for faithful generation
• Pairs with evaluation metrics like Knowledge F1
• Demonstrates that faithfulness and fluency are not mutually exclusive

FaithDial is a direct derivative of the Wizard of Wikipedia dataset, which is known to contain a high proportion of hallucinated responses. By systematically correcting these dialogues, FaithDial provides a controlled study in how models deviate from knowledge sources and how to enforce grounding constraints during generation.

• Source: Wizard of Wikipedia dialogues
• Correction process: Identify unsupported spans, rewrite with source entailment
• Preserves conversational diversity while eliminating fabrication

FaithDial is frequently used to train the factual verification components of guardrails systems like NeMo Guardrails. By learning from FaithDial's entailment patterns, these systems can intercept LLM outputs in real-time, validate claims against retrieval sources, and block or rewrite hallucinated statements before they reach the user.

• Trains factual verification rails for production systems
• Enables real-time hallucination interception
• Complements Retrieval-Augmented Verification architectures

The source dataset from which FaithDial was derived. WoW is a knowledge-grounded dialogue benchmark where one participant acts as a 'wizard' using Wikipedia to inform responses. However, the original dataset contains hallucinated responses where wizards ignored or contradicted retrieved knowledge. FaithDial corrects these unfaithful turns, making it a critical resource for studying the gap between human dialogue and factual grounding.

A composite metric calculating the harmonic mean between the precision and recall of factual knowledge units. For FaithDial-trained models, Knowledge F1 measures the balance between:

• Factual Precision: Are generated statements correct relative to the source?
• Factual Recall: How much of the source knowledge is reflected in the output? This ensures models don't just stay silent to avoid hallucination but actively incorporate grounding information.

A prompting technique where an LLM drafts, verifies, and corrects its own output. The model generates verification questions, answers them independently, and produces a final response consistent with verified facts. This mirrors FaithDial's methodology at inference time—the dataset's human annotators performed a similar fact-check-then-rewrite loop, making CoVe a natural complement for models fine-tuned on FaithDial.

A special token sequence used in FaithDial to explicitly signal the start of grounding information. When models are trained on FaithDial, they learn to condition responses on this delimiter, creating a clear boundary between retrieved knowledge and generated dialogue. This architectural pattern enables attribution tracing—every claim after the token must be supported by the preceding knowledge segment.