Probabilistic Relevance Framework

The foundational rule stating that optimal retrieval effectiveness is achieved when documents are ranked in decreasing order of their probability of relevance to the query. The PRP assumes that the relevance of one document is independent of others, providing a clear decision theory for search systems. Key insight: ranking by P(Relevant|Document) minimizes expected search costs.

A classic probabilistic model that treats documents as binary vectors indicating term presence or absence. The BIM assumes term independence and estimates relevance odds based on term distributions in relevant and non-relevant sets. Core formula: the relevance weight for a term is log(p(1-q) / q(1-p)), where p is the probability of the term appearing in a relevant document and q in a non-relevant one.

A method for estimating term weights without relevance information by approximating the probability of term occurrence. It assumes that non-relevant documents approximate the whole collection and adds a 0.5 smoothing factor to avoid zero probabilities. This weighting directly evolves into the Inverse Document Frequency (IDF) component used in BM25.

A statistical model proposing that term frequency distributions follow a mixture of two Poisson distributions: one for 'elite' documents where the term is topical, and one for documents where it appears by chance. BM25's saturation function is an analytical approximation of this model, capturing the diminishing returns of repeated term occurrences.

An iterative process where user judgments on initial results refine the probabilistic model. By updating the estimates of p and q based on identified relevant documents, the system reformulates the query vector to pull in more relevant items. This closed-loop mechanism is the theoretical basis for modern pseudo-relevance feedback techniques.

The concept that for a given topic, documents can be divided into an 'elite' set where a term is semantically about the topic, and a non-elite set where it appears incidentally. The probabilistic framework models this latent variable to distinguish between topical usage and spurious occurrences, informing how term saturation is mathematically handled.

The dominant implementation of the Probabilistic Relevance Framework, ranking documents by estimating relevance using term frequency saturation, inverse document frequency, and document length normalization. It improves upon TF-IDF by preventing term frequency from dominating scores and adjusting for document verbosity.

The theoretical foundation for BM25's IDF component. This method estimates the probability of a term appearing in relevant versus non-relevant documents without requiring explicit relevance judgments, using collection-wide statistics as a proxy for relevance information.

A mathematical component within BM25 that models the diminishing returns of term frequency. The first occurrence of a term contributes significantly to relevance, but additional occurrences add progressively less weight, preventing keyword stuffing from dominating rankings.

Two free parameters that tune BM25 behavior for specific collections:

• k1: Controls term frequency saturation speed (typical range: 1.2–2.0)
• b: Controls document length normalization strength (0 = no normalization, 1 = full scaling) Default values of k1=1.2 and b=0.75 work well for most text corpora.

The data structure that enables efficient probabilistic retrieval at scale. Maps each unique term to a postings list containing document IDs and term frequency metadata, allowing the scoring function to access only documents containing query terms rather than scanning the entire collection.

A retrieval paradigm where documents and queries are represented as high-dimensional vectors with mostly zero values, where non-zero dimensions correspond to explicit term weights. This enables exact lexical matching via inverted indices, contrasting with dense neural retrieval approaches.