b Parameter

The k1 parameter controls the scale of the term frequency saturation curve. While b governs document length normalization, k1 governs how quickly the score for a term plateaus with additional occurrences.

• Interaction: A high k1 amplifies the raw term frequency, making the length normalization applied by b more impactful.
• Typical Range: Set between 1.2 and 2.0. A value of 0 ignores term frequency entirely, relying only on IDF.

This is the core mechanism that the b parameter directly modulates. It adjusts a document's relevance score to prevent longer documents from having an unfair advantage simply because they contain more words.

• b=0: No normalization. Raw term frequency is used, favoring long documents.
• b=1: Full normalization by the ratio of the collection's average length. Short, dense documents are heavily favored.
• Pivot Point: The b parameter effectively sets the pivot document length where normalization switches from penalizing to boosting.

BM25F extends the base algorithm to handle structured documents with multiple fields (e.g., title, body, abstract). It computes a weighted aggregate of per-field BM25 scores.

• Field-Specific b: Each field can have its own b parameter to model different length dynamics. A title field often uses b=0 (no normalization) since all titles are short.
• Weighting: Field weights (boost factors) are multiplied by the normalized score, allowing precise tuning of signal importance.

Since version 5.0, Elasticsearch uses BM25 as the default similarity. The default parameters are k1=1.2 and b=0.75.

• Practical Tuning: The b=0.75 default provides a balanced approach, partially normalizing for length. For short-form content (tweets, product names), lowering b to 0.25–0.5 is common.
• Configuration: These parameters are set at the index level and can be overridden per field in the mapping.

BM25 is the quintessential sparse retrieval algorithm, relying on explicit lexical matching. The b parameter's length normalization is a heuristic to solve a problem that dense retrieval models (DPR, Sentence-BERT) handle differently.

• Sparse (BM25): Uses b to manually correct for length bias in bag-of-words vectors.
• Dense (Embeddings): Length normalization is often an emergent property of the training objective or handled via cosine similarity on fixed-dimensional vectors.
• Hybrid Fusion: In modern systems, BM25 scores (tuned with b) are combined with dense vector scores via Reciprocal Rank Fusion (RRF).

Pseudo-Relevance Feedback (PRF) assumes the top-K documents from an initial BM25 retrieval are relevant and extracts expansion terms from them. The b parameter critically influences which documents enter this feedback loop.

• Bias Amplification: If b is poorly tuned, the initial result set may be biased toward long or short documents, causing PRF to extract non-representative expansion terms.
• Stability: A well-calibrated b ensures the top-K documents are genuinely the most relevant, providing a stable foundation for automatic query expansion.