Cross-Lingual Re-Ranking

Cross-lingual re-ranking separates retrieval into candidate generation and precision scoring stages. The first stage uses a fast bi-encoder (e.g., mDPR or LaBSE) to encode queries and documents independently into a shared multilingual vector space, retrieving top-k candidates via approximate nearest neighbor search. The second stage employs a cross-encoder that processes the query-document pair jointly through full transformer attention, enabling fine-grained semantic comparison across languages. This architecture balances the speed of dense retrieval with the accuracy of deep cross-lingual understanding.

Unlike bi-encoders that produce independent embeddings, a cross-encoder concatenates the query and candidate document into a single input sequence: [CLS] query [SEP] document [SEP]. The model applies full self-attention across all tokens, allowing the query terms to directly attend to document terms in both languages. This joint encoding captures subtle cross-lingual relationships—such as paraphrases, entailments, and partial matches—that independent vector comparisons miss. The output is a relevance score between 0 and 1, used to reorder the initial candidate set.

Training a cross-lingual re-ranker often leverages knowledge distillation from a powerful multilingual teacher model. A large model like XLM-RoBERTa is fine-tuned on cross-lingual relevance judgments, then its output probabilities are used to train a smaller, faster student model. The distillation loss minimizes the KL divergence between teacher and student score distributions. This approach preserves cross-lingual transfer capabilities while reducing inference latency by 5-10x, making re-ranking viable for production systems serving queries across 100+ languages.

Effective re-ranker training requires hard negatives—documents that are topically similar to the query but not relevant. In cross-lingual settings, hard negatives are mined by:

• Retrieving high-scoring candidates from the first-stage bi-encoder that are in a different language than the query
• Using round-trip translation to generate plausible but incorrect document pairs
• Mining cross-lingual paraphrases that share vocabulary but differ in meaning These challenging examples force the cross-encoder to learn fine-grained semantic distinctions beyond surface-level lexical overlap.

A cross-lingual re-ranker trained exclusively on English relevance data can zero-shot transfer to unseen languages. This works because multilingual encoders like XLM-RoBERTa map semantically equivalent queries and documents to similar representation regions regardless of language. The re-ranker learns to recognize relevance patterns—such as entailment and contradiction—that generalize across language boundaries. Evaluation on CLEF and TREC multilingual benchmarks shows that zero-shot re-rankers achieve 85-95% of the performance of fully supervised in-language models.

Cross-lingual re-rankers employ two scoring paradigms:

• Pointwise: Each query-document pair is scored independently, producing a relevance probability. Simple but ignores inter-document relationships.
• Listwise: The model processes the entire candidate list jointly, optimizing a ranking metric like NDCG directly. Listwise approaches (e.g., RankT5, monoT5) capture relative document quality and diversity, preventing near-duplicate cross-lingual translations from dominating top positions. Listwise methods typically outperform pointwise by 3-8% on NDCG@10 for multilingual retrieval tasks.