Conversational Reranking

Unlike a bi-encoder that encodes the query and document separately, a cross-encoder processes the concatenated query-document pair through a transformer model simultaneously. This allows full token-level attention between the query and document, producing a relevance score that captures nuanced semantic relationships. The trade-off is computational cost: cross-encoders are too slow for initial retrieval over millions of documents but excel at reranking a small candidate set (top 100-1000). Common implementations include MiniLM and monoT5 models fine-tuned on MS MARCO passage ranking data.

The first-stage retrieval mechanism that conversational reranking depends on. A bi-encoder independently encodes the query and each document into dense vector embeddings, then computes relevance via cosine similarity. This enables fast approximate nearest neighbor (ANN) search across millions of documents using vector databases like Pinecone or Weaviate. Bi-encoders sacrifice some accuracy for speed, which is why a second-stage cross-encoder reranker is applied to refine the top results.

The full dialogue history that a reranker must consider, not just the current user utterance. In multi-turn conversations, queries often contain anaphora (pronouns like 'it' or 'that') and ellipsis (omitted words) that only make sense with prior context. A conversational reranker concatenates the current query with preceding turns—or a conversation summary—before scoring documents. This prevents the system from retrieving irrelevant documents when a user says 'Tell me more about the second one' without restating the topic.

A foundational bi-encoder architecture from Facebook AI Research that uses dual BERT encoders trained on question-passage pairs. DPR demonstrated that dense retrieval could outperform traditional BM25 sparse retrieval by capturing semantic similarity rather than keyword overlap. In a conversational reranking pipeline, DPR often serves as the first-stage retriever, generating the candidate set that a cross-encoder then reorders. The DPR training objective maximizes the dot product between relevant query-passage pairs while minimizing it for negatives using in-batch negatives.

A preprocessing step that resolves conversational dependencies before retrieval and reranking. When a user asks 'What about pricing?' after discussing features, a query reformulation model rewrites this into a standalone query like 'What is the pricing for the enterprise plan?' This decontextualized query can then be used by both the bi-encoder retriever and the cross-encoder reranker. Techniques include fine-tuned T5 models trained on the CANARD dataset or using the LLM itself to generate a context-independent reformulation.

The benchmark dataset from Microsoft that has become the standard for training and evaluating reranking models. It contains over 1 million real Bing queries paired with human-annotated relevant passages. The dataset includes both a full ranking task (reorder 1000 candidates) and a binary relevance task. Most modern cross-encoder rerankers, including monoT5 and RankLLaMA, are fine-tuned on MS MARCO before being adapted to conversational search scenarios. Performance is measured using MRR@10 and nDCG@10.