Full-Attention Scoring

The query and candidate document are concatenated into a single sequence: [CLS] query [SEP] document [SEP]. This unified input is fed through the transformer simultaneously. Because the model processes both strings as one contiguous context, the self-attention mechanism can establish direct token-to-token relationships across the query-document boundary from the very first layer.

The core of full-attention scoring is the attention matrix. For a query of length m and a document of length n, the model computes an (m+n) x (m+n) attention grid. This allows the model to weigh specific alignments, such as matching the query token 'Python' against document tokens 'snake' (low weight) versus 'programming' (high weight), capturing exact match signals and lexical overlap that Bi-Encoders miss.

Unlike static embedding comparisons, full-attention scoring is context-dependent. The representation of a document token shifts based on the query it is paired with. For example, the word 'bank' in a document will be contextualized differently when the query is 'financial institutions' versus 'river erosion'. This dynamic adaptation is impossible in Bi-Encoder architectures where document vectors are pre-computed and frozen.

After the final transformer layer, the model must produce a single scalar relevance score. The most common method uses the [CLS] token representation, which has aggregated information from the entire sequence. This vector is passed through a linear classification head to output a logit, which is then converted to a probability via a sigmoid function, representing the likelihood of relevance.

The expressiveness of full-attention scoring comes at a significant cost. The self-attention operation scales quadratically with the combined sequence length: O((m+n)²). This makes it computationally prohibitive to apply against millions of documents in real-time. Consequently, it is strictly reserved for a re-ranking phase, applied only to the top-k candidates (e.g., top 100) retrieved by a fast Bi-Encoder or lexical system.

Full-attention scoring can be applied in two modes:

• Pointwise: A single query-document pair is scored independently, producing an absolute relevance probability.
• Pairwise: Two documents are scored simultaneously against the same query to predict which is more relevant. This is exemplified by DuoBERT, which uses a [CLS] query [SEP] docA [SEP] docB [SEP] input format to learn fine-grained comparative distinctions.

The granular semantic matching enabled by the Cross-Encoder's self-attention mechanism. Unlike Bi-Encoders that compress text into a single vector, full-attention scoring allows the model to weigh specific term alignments between query and document tokens.

• Captures exact match signals and synonym detection simultaneously
• Enables the model to learn that 'car' attending to 'automobile' is a strong match
• Identifies negation patterns where 'not good' reverses sentiment
• Computationally expensive: O(n²) complexity relative to combined sequence length

The fundamental operation where a query string and a candidate passage are concatenated into a single input sequence and passed through a transformer. The model processes the joint representation through multiple layers of self-attention.

• Input format: [CLS] query [SEP] document [SEP]
• The [CLS] token aggregates the full cross-attention signal into a single relevance embedding
• A linear classification head projects this embedding to a scalar relevance score
• Enables the model to condition document understanding directly on query context

A multi-stage retrieval design where full-attention scoring is the final precision filter. The pipeline progressively narrows candidates from millions to a final ranked list.

• Stage 1: Sparse retrieval (BM25) or dense vector search retrieves top-1000 candidates
• Stage 2: Lightweight Bi-Encoder reranks to top-100
• Stage 3: Cross-Encoder with full-attention scoring produces the final top-10 ranking
• Balances the latency-relevance trade-off by reserving expensive full-attention for only the most promising candidates

Techniques to reduce the computational cost of full-attention scoring to meet real-time search latency budgets. Cross-Encoders are inherently slower than Bi-Encoders due to the inability to pre-compute document representations.

• INT8 quantization reduces model precision with minimal accuracy loss
• ONNX Runtime with graph optimizations and operator fusion accelerates transformer inference
• Dynamic batching groups multiple query-document pairs for parallel GPU processing
• Knowledge distillation transfers full-attention scoring patterns to a faster student Bi-Encoder

A training data strategy that selects challenging negative samples to teach the full-attention mechanism fine-grained discrimination. Standard random negatives are too easy and fail to push the model to learn subtle distinctions.

• Retrieves top-scoring documents from a BM25 or dense retriever that are not relevant
• Forces the Cross-Encoder to distinguish between lexically similar but semantically irrelevant passages
• Critical for learning to detect false exact matches where keywords overlap but meaning diverges
• Often combined with in-batch negatives for computational efficiency during contrastive training

The process of adjusting raw Cross-Encoder logits so that scores reflect true empirical relevance probabilities. Raw transformer outputs are often overconfident and poorly calibrated.

• Platt scaling fits a logistic regression model on validation set scores to map logits to probabilities
• Temperature scaling divides logits by a learned scalar parameter to soften the softmax distribution
• Calibrated scores enable meaningful score thresholds across different queries and domains
• Essential for downstream systems that fuse Cross-Encoder scores with other relevance signals