Query Performance Prediction (QPP)

Estimate query difficulty before any search is executed by analyzing the query's linguistic and statistical properties. This approach is computationally cheap, adding negligible latency.

• Query Length & IDF: Short, common-word queries often perform worse than specific, rare-word queries.
• Clarity Score: Measures the divergence between the query language model and the collection language model. A high divergence suggests a focused, well-defined topic.
• Query Scope: Quantifies the generality of a query by analyzing the size of the retrieved set in an inverted index without full scoring.

Analyze the result list after an initial retrieval to estimate query performance. These methods are more accurate but incur higher computational cost.

• Clarity Score (Post-Retrieval): Computes the KL-divergence between the language model of the top-k retrieved documents and the collection model.
• Weighted Information Gain (WIG): Measures the divergence of the top-k document scores from the mean score of the entire corpus.
• Normalized Query Commitment (NQC): Evaluates the variance of document scores in the result list; a high variance indicates a clear distinction between relevant and non-relevant documents.

A specific post-retrieval predictor that measures the sensitivity of the ranked list to perturbations. It is based on the idea that a robust query will have a stable result set even when the query or documents are slightly altered.

• Mechanism: Reweights documents based on their similarity to the centroid of the top-ranked results.
• Interpretation: A high robustness score indicates a coherent result set and a well-performing query. A low score suggests the top documents are thematically diffuse.

Examines the statistical shape of the retrieval scores for the top-k documents. The underlying assumption is that the distribution of scores for a high-performing query differs significantly from a poorly-performing query.

• Skewness: Measures the asymmetry of the score distribution. A negatively skewed distribution (scores clustered near the top) often signals high performance.
• Kurtosis: Measures the 'tailedness' of the distribution.
• Variance (NQC): As noted, high variance among top scores is a strong positive signal.

The primary architectural purpose of QPP is to act as a gate for alternative retrieval pipelines. A predicted low-performance score triggers a fallback.

• Threshold-Based Trigger: If QPP_Score < Threshold, initiate a fallback.
• Fallback: Query Expansion: Automatically trigger Pseudo-Relevance Feedback (PRF) or HyDE to enrich the query.
• Fallback: Query Clarification: Ask the user an open-ended or facet-based clarifying question to resolve ambiguity.
• Fallback: Sparse-Dense Hybridization: If the initial dense retrieval fails, fall back to a BM25 sparse search.

Modern approaches that use fine-tuned language models to directly predict a query's performance metric, such as Average Precision (AP) or nDCG, without relying on hand-crafted statistical features.

• qppBERT: A BERT-based model fine-tuned on pairs of queries and their historical performance data.
• Fine-Tuning Data: Requires a corpus with relevance judgments to train the model to map query embeddings to a performance score.
• Cross-Encoder Approach: A cross-encoder can be trained to predict the relevance of a query to a pseudo-document, with the score serving as the QPP estimate.

Estimates query effectiveness before any search is executed. This approach analyzes the query's linguistic properties, such as specificity, clarity, and syntactic complexity, to predict performance. A high score suggests a well-formed query, while a low score triggers proactive interventions like query rewriting or automatic synonym expansion to avoid a wasted retrieval cycle. It is computationally cheap but lacks index-level context.

Evaluates query effectiveness after an initial retrieval set is obtained. It analyzes the coherence and statistical properties of the top-k result list, such as the clarity score (divergence between the result set and the collection language model) or the weighted information gain. This method is more accurate because it has real data, but it adds latency. A poor score typically triggers pseudo-relevance feedback or a fallback to a clarification dialog.

A foundational post-retrieval QPP metric that measures the Kullback-Leibler divergence between the language model of the top-retrieved documents and the background collection model. A high divergence indicates a focused, distinct result set, suggesting a high-performance query. A low divergence implies the results are generic, mirroring the overall corpus, which signals a likely retrieval failure.

The operational application of QPP in production systems. It involves continuously tracking QPP scores over time to detect semantic drift in user behavior or document collections. A sudden drop in average QPP for a specific query cluster can automatically alert engineers to re-index content, update embedding models, or refresh synonym rings without waiting for user complaint tickets.

A QPP variant that measures a query's sensitivity to document perturbations. It tests stability by randomly injecting noise or removing top documents from the index. A query is deemed robust if its retrieval set remains consistent despite these changes. This metric is critical for conversational AI systems where minor rephrasings should not cause catastrophic changes in the retrieved evidence.

A post-retrieval predictor that calculates the average difference between the relevance score of the top-ranked document and every other document in the result set. A high WIG implies a dominant, clearly superior document exists, predicting high precision. A low WIG suggests many documents have similar scores, indicating ambiguity and a higher risk of ranking errors.