Candidate Pool Merging

The foundational operation of candidate pool merging is a set union on document IDs returned by independent retrieval systems. Since sparse (BM25) and dense (vector) indexes often retrieve overlapping documents, deduplication ensures each candidate appears only once in the merged pool. This is typically performed using a hash set or Bloom filter on the document's unique identifier, preventing redundant scoring and ensuring the downstream re-ranker processes a clean, minimal candidate set.

Controlling the cardinality of the merged candidate pool is critical for latency budgeting. A naive union of top-K results from N retrievers can produce up to N*K candidates, overwhelming a computationally expensive re-ranker.

• Fixed Cardinality: Truncate the merged set to a hard limit (e.g., 1000 documents).
• Score-Based Pruning: Discard candidates below a minimum BM25 or vector similarity threshold before merging.
• Tiered Retrieval: Retrieve a larger pool from fast indexes and a smaller, high-precision pool from slower systems.

During merging, the raw relevance scores from each retrieval subsystem must be preserved and associated with their respective document IDs. These scores serve as input features for the subsequent fusion algorithm (e.g., RRF or weighted sum). A merged candidate record typically stores a vector of scores: {doc_id: "A", bm25_score: 12.4, dense_score: 0.89}. If a document was retrieved by only one system, the missing score is imputed as a default minimum or zero before normalization.

To optimize performance, candidate pool merging often incorporates early termination heuristics. If a sufficient number of high-confidence candidates are identified early in the merge process, the system can stop pulling results from slower indexes.

• WAND Operator: Adapts the classic Weak-AND algorithm to stop scoring when a document's combined score cannot enter the top-K.
• Confidence Thresholds: If the top-10 candidates from the dense retriever all have cosine similarity > 0.95, the system may skip the sparse retriever entirely.

Candidate pool merging occurs at a distinct architectural point: before the fusion algorithm runs. This is a critical distinction from post-fusion merging.

• Pre-Fusion Merge: Combine IDs first, then apply RRF or CombSUM to the unified pool. This is the standard approach for most hybrid search systems.
• Post-Fusion Merge: Each retriever produces a fully ranked list, and the lists are interleaved or combined only at the final ranking stage. This is less common and risks losing candidates that one system ranked poorly.

Candidate pool merging must respect pre-retrieval metadata filters. If a user's query includes a hard constraint (e.g., year > 2020), this filter is typically pushed down to each retriever's index scan. The merged pool then contains only documents that satisfy the constraint. A common failure mode is post-filtering during merge, where a document retrieved by one system is discarded because it fails a metadata check, leading to an unexpectedly small candidate pool and potential empty result sets.