Answer Correctness

This component measures whether all factual claims in the generated answer are entirely supported by the provided source context. It is the foundation of correctness, ensuring the model does not hallucinate or contradict its sources.

• Evaluation Method: Typically involves a Natural Language Inference (NLI) model or a fine-tuned classifier to check if the answer can be logically inferred from the context.
• Key Distinction: Separate from answer relevance; an answer can be on-topic but factually incorrect.
• Example: If the context states "The company was founded in 2010," a correct answer must not say "founded in 2012."

This assesses the degree of meaning equivalence between the generated answer and a human-verified reference (ground truth). It moves beyond exact string matching to evaluate if the same information is conveyed.

• Primary Metrics: BERTScore and Semantic Similarity (using sentence embeddings) are standard tools. They are more robust than ROUGE or BLEU for paraphrase-heavy domains.
• Use Case: Critical for evaluating answers that can be phrased in multiple valid ways. A score of 0.95 indicates near-identical meaning, while 0.7 may indicate missing or distorted key details.

Evaluates if the answer fully addresses the query's intent and includes all key information present in the ground truth, without introducing extraneous or off-topic details.

• Completeness: Measures the proportion of key information points from the ground truth that are present in the generated answer. Missing a critical point (e.g., omitting "in 2010" from a founding date) reduces correctness.
• Relevance: Ensures the answer does not contain unsolicited information. An answer adding unrelated facts about a company's product when asked only for its founding year is less correct.
• Quantification: Often measured via Precision and Recall over information units, combined into an F1 Score.

For systems that provide citations, correctness requires that every factual statement is accurately anchored to the specific source passage that supports it. This enables verification and is a proxy for faithfulness.

• Source Citation Precision: The percentage of provided citations that are accurate and support the adjacent claim. A low score indicates spurious citations.
• Source Citation Recall: The percentage of claims in the answer that should be cited (i.e., are verifiable facts) and actually have a citation. A low score indicates unsupported assertions.
• Engineering Impact: High citation accuracy is non-negotiable for enterprise and legal RAG applications, forming the basis for algorithmic trust.

The most robust method for reference-free evaluation assesses the logical relationships within the triad of user Query, retrieved Context, and generated Answer. This framework underpins tools like RAGAS.

• Faithfulness (Answer <- Context): Is the answer supported by the context?
• Answer Relevance (Query <- Answer): Does the answer address the query?
• Context Relevance (Query <- Context): Was the retrieved context useful for the query?
• Composite Score: A weighted combination of these scores yields a holistic Answer Correctness metric without needing a pre-written ground truth, enabling scalable evaluation.

Answer Correctness is intrinsically dependent on upstream retrieval quality. A perfect generator cannot produce a correct answer if the necessary information was not retrieved.

• Key Dependency: High Answer Correctness scores are only possible with high Retrieval Recall (to get the needed facts) and Context Relevance (to avoid noise).
• Diagnostic Use: Low correctness with high faithfulness suggests a retrieval failure—the model is faithful to poor context. This directs engineering effort to improve embedding models or chunking strategies.
• End-to-End Metric: Therefore, Answer Correctness serves as the ultimate end-to-end metric for the entire RAG pipeline, from query understanding to final generation.

Answer Faithfulness measures the extent to which a generated answer is factually consistent with and supported by the provided source context. It is a critical sub-component of Answer Correctness, focusing on whether the model 'hallucinates' information not present in the sources.

• A high faithfulness score indicates the answer is fully grounded in the retrieved documents.
• This metric is often evaluated by checking if claims in the answer can be directly attributed to statements in the context.
• It is a prerequisite for factual correctness; an unfaithful answer cannot be correct.

Answer Relevance evaluates how directly and completely a generated answer addresses the original query, independent of its factual correctness. It assesses the model's ability to stay on-topic.

• An answer can be relevant (directly addresses the query) but incorrect (contains factual errors).
• Conversely, an answer can be factually faithful to the source but irrelevant to the user's question.
• This metric ensures the system provides useful, focused responses, which is a necessary condition for a correct answer to be valuable.

Grounding Score is a metric that quantifies the degree to which a model's generated output is substantiated by specific, attributable information from its provided source materials. It is closely related to faithfulness but often involves a finer-grained, claim-by-claim attribution analysis.

• High grounding means each factual statement in the answer can be linked to a specific passage in the source context.
• This is crucial for building trust and enabling verification in enterprise applications.
• Tools like RAGAS and TruLens implement automated methods to compute grounding or faithfulness scores.

Context Relevance assesses the degree to which the text passages retrieved and provided to the language model are pertinent and useful for answering the specific query. It evaluates the quality of the retrieval step.

• If the retrieved context is irrelevant, even a perfect language model cannot generate a correct answer.
• This metric measures the signal-to-noise ratio in the context provided to the generator.
• High context relevance is a foundational requirement for achieving high answer correctness.

Semantic Similarity quantifies the likeness in meaning between two pieces of text, such as a generated answer and a ground truth reference. It is a common automated proxy for correctness when human evaluation is not feasible.

• Unlike Exact Match (EM), it uses embeddings from models like Sentence-BERT to understand paraphrases and conceptual equivalence.
• Metrics like BERTScore leverage this principle, comparing the contextual embeddings of tokens in the candidate and reference texts.
• While useful, it should be complemented with faithfulness metrics, as a semantically similar answer could still introduce unsupported details.