Attribution Scoring

Attribution scoring operates at the most granular level, mapping specific spans of generated text to precise segments in the source document. Unlike document-level retrieval, this method uses token probability differentials to measure how much a source passage influenced a particular word. A high score indicates that the generated statement is a direct, faithful paraphrase of the source, not a probabilistic confabulation. This is often implemented using gradient-based feature attribution or input perturbation techniques to isolate causal links between input evidence and output text.

A raw attribution score is meaningless without normalization. This characteristic involves calibrating the score against a baseline of known faithful and hallucinated examples to produce a probability between 0 and 1. A score of 0.95 means the system is 95% confident the statement is entailed by the source. This process corrects for a model's overconfidence by applying temperature scaling or conformal prediction wrappers, ensuring the final metric is a reliable indicator of factual risk for downstream human review workflows.

In multi-document legal reasoning, a single conclusion might require support from multiple sources. This characteristic evaluates whether a statement's attribution is consistent across all cited documents. The scoring algorithm performs a contradiction check by comparing the embedding similarity of the generated claim against all referenced source segments. A statement with high attribution to one source but direct contradiction with another receives a penalized, lower composite score, flagging it as a normative conflict requiring human resolution.

Attribution scoring directly powers automated abstention mechanisms. By setting a strict operational threshold (e.g., 0.85), the system is programmed to refuse to generate an answer if no source passage meets the minimum attribution score. This prevents the model from falling back to its internal parametric knowledge and hallucinating a plausible-sounding but unsupported legal conclusion. This characteristic transforms the metric from a passive evaluation tool into an active safety guardrail that enforces strict knowledge grounding.

The primary utility of the score is to generate a human-readable audit trail. Each scored statement is output with a heatmap overlay or a direct hyperlink to the exact source paragraph and line numbers. This characteristic supports the legal doctrine of verifiable provenance, allowing a reviewing attorney to instantly validate the AI's reasoning by comparing the generated text side-by-side with the source material. This transparent mapping from output to origin is the core defense against the 'black box' objection in high-stakes litigation.

This characteristic specifically measures the model's ability to ignore its pre-training data and rely solely on the provided context. The scoring function penalizes the presence of extraneous entities or legal tests that appear in the output but are absent from the input documents. By using a Natural Language Inference (NLI) entailment model as a judge, the system can detect when a generated statement is factually consistent with the source but introduces outside knowledge, resulting in a low adherence score despite surface-level accuracy.

The foundational capability of an AI system to pinpoint the exact origin of information. Unlike a simple citation, source attribution provides a transparent audit trail from a generated output back to the raw source text segment. This is the prerequisite for calculating an attribution score, as you cannot score what you cannot trace. In legal AI, this often involves mapping a sentence to a specific paragraph or clause in a contract or judicial opinion.

An automated verification process that confirms whether every factual claim in a generated text is explicitly supported by the provided source document. It acts as a critical guardrail against hallucination. While attribution scoring quantifies the degree of linkage, groundedness detection is the binary or probabilistic classifier that determines if a claim is entailed by the source at all. It is the primary mechanism for enforcing context adherence.

Two metrics that provide a granular view of citation integrity. Citation Recall measures the proportion of factual claims that are correctly supported by a citation, answering 'Did we cite everything we should have?' Citation Precision measures the proportion of provided citations that genuinely support their associated claim, detecting fabricated or irrelevant references. Together, they decompose the broader attribution score into its constituent failure modes.

A classification task that determines the logical relationship between a premise (source text) and a hypothesis (generated statement). The three possible labels are entailment, contradiction, or neutral. This is the core computational engine behind many attribution scoring systems. A high attribution score requires a high rate of entailment between the generated legal analysis and the cited source segments, ensuring no contradictory or unsupported inferences are made.

A quantitative evaluation framework that measures the factual consistency of a generated summary or answer relative to the source material. It identifies contradictions and unsupported fabrications. An attribution score is a specific type of faithfulness metric focused on the provenance link, whereas a general faithfulness metric may also penalize omissions or incorrect aggregation of source facts that are not strictly hallucinated.

A faithfulness metric that evaluates whether a model's response is strictly derived from the user-provided context. It penalizes the introduction of external knowledge or assumptions not present in the input. In legal AI, this is paramount; a model must not supplement a contract analysis with its general knowledge of contract law unless explicitly instructed. Attribution scoring directly measures this adherence by requiring a verifiable link for every assertion.