Citation Recall

Citation Recall is calculated as a strict ratio: the number of factual claims with a correct citation divided by the total number of factual claims in the generated text. A claim is a discrete, verifiable statement of fact. A correct citation is one where the referenced source document explicitly and accurately supports the claim. This metric ignores irrelevant or hallucinated citations, which are measured separately by Citation Precision.

Citation Recall and Citation Precision are complementary metrics that together define Attribution Fidelity. Recall measures coverage—did you cite enough? Precision measures accuracy—were your citations correct?

• High Recall, Low Precision: The model cites many sources, but some are irrelevant or hallucinated.
• Low Recall, High Precision: The model cites few sources, but every single one is perfectly accurate.
• Target State: High Recall and High Precision, indicating comprehensive and accurate source grounding.

Citation Recall can be evaluated at multiple levels of granularity, each revealing different failure modes:

• Claim-Level: The standard approach. Each atomic fact must have its own citation.
• Sentence-Level: A coarser measure where an entire sentence is considered cited if at least one claim within it is supported.
• Document-Level: The weakest measure, only checking if a general reference to a source exists, regardless of whether it supports the specific text.

Fine-grained, claim-level evaluation is the gold standard for detecting Attribution Drift and Null Attribution.

Citation Recall is a direct, operationalized counter-measure to factual hallucination. A low Citation Recall score is a leading indicator of Hallucination Risk. If a model generates 10 factual claims but only cites 4, the remaining 6 are unsupported assertions. These uncited claims are the primary candidates for hallucination. Monitoring Citation Recall in production provides a quantitative signal for the trustworthiness of a Retrieval-Augmented Generation (RAG) pipeline.

Manual evaluation of Citation Recall does not scale. The standard automated approach uses a Natural Language Inference (NLI) model as a judge. The process is:

1. Decompose the generated text into atomic claims.
2. For each claim, retrieve the text of its cited source document.
3. Feed the premise (source text) and hypothesis (claim) to an NLI model.
4. The NLI model classifies the pair as entailment, contradiction, or neutral.

Only claims with an entailment classification count toward the numerator of the Recall score.

Optimizing a system for perfect Citation Recall can paradoxically lower Citation Precision. A model instructed to 'cite everything' may begin attaching citations to subjective statements, common knowledge, or syntactical filler, or it may cite a source that is topically related but does not actually support the specific claim. This is known as Attribution Drift. The engineering goal is to maximize both metrics simultaneously, a challenge addressed by advanced Retrieval-Augmented Attribution architectures.

Measures the proportion of provided citations that are correct and relevant. While Citation Recall asks 'did we cite all claims?', Citation Precision asks 'are our citations actually right?' A system with high recall but low precision floods users with irrelevant or hallucinated references.

• Formula: (Number of correct citations) / (Total number of citations provided)
• Failure mode: Citing a source that doesn't contain the claimed fact
• Trade-off: Optimizing for recall alone often degrades precision

A deeper metric that evaluates whether a citation accurately represents the information in the referenced source, not just whether the source exists. A citation can be precise (pointing to a real document) but have low fidelity if it mischaracterizes what the source actually says.

• Evaluates: Semantic alignment between claim and source text
• Detection method: NLI models comparing generated claim to cited passage
• Critical for: Legal, medical, and financial applications where misrepresentation carries liability

The proportion of generated factual claims that are entirely unsupported by any verifiable source. This is the inverse complement to Citation Recall—a high hallucination rate indicates the model is fabricating facts rather than grounding them in retrieved evidence.

• Relationship to Recall: Hallucination Rate ≈ 1 - Citation Recall (simplified)
• Detection: Requires human evaluation or automated fact-checking pipelines
• Mitigation: RAG architectures with strict attribution requirements reduce this metric

A fine-grained attribution technique that traces specific word sequences in generated text back to exact source documents. Unlike document-level citation, n-gram provenance identifies which specific sentences or passages contributed to each claim.

• Granularity: Operates at the phrase level, not the document level
• Implementation: Uses overlap detection between generated text and retrieval corpus
• Use case: Debugging attribution failures and auditing model outputs for plagiarism

An architectural pattern where an external trusted knowledge source is queried in real-time to fact-check claims before they are presented to the user. This closes the loop on Citation Recall by actively verifying, not just passively citing.

• Workflow: Generate claim → Retrieve evidence → Verify alignment → Cite or suppress
• Key components: Fact-checking NLI model, authoritative knowledge base
• Outcome: Higher Citation Recall and Precision simultaneously

Measures how well a model's internal probability of correctness aligns with its actual accuracy. A well-calibrated model knows when it's likely wrong and can refuse to answer or flag uncertainty—preventing low-confidence claims from entering the Citation Recall calculation.

• Metric: Expected Calibration Error (ECE)
• Benefit: Reduces the burden on citation systems by filtering uncertain outputs
• Implementation: Temperature scaling, conformal prediction