Mention-Ranking Model

Unlike mention-pair models that classify each antecedent-mention pair independently, mention-ranking models evaluate all candidates simultaneously for a given mention. The model computes a compatibility score for every candidate antecedent using a learned scoring function—typically a biaffine attention mechanism—and selects the highest-scoring candidate. This architecture naturally handles the constraint that each mention has at most one antecedent while allowing the model to compare candidates against each other implicitly through the softmax normalization over the candidate set.

The foundation of modern mention-ranking models rests on dense span representations rather than hand-engineered linguistic features. Each candidate span is encoded by concatenating or attending over the hidden states of its boundary tokens, typically using architectures like SpanBERT that are pre-trained specifically for span-level tasks. The representation captures:

• The span's internal syntactic structure
• Its semantic content
• Contextual information from surrounding discourse This end-to-end learning eliminates the need for the brittle head-finding heuristics and syntactic parsers that dominated earlier deterministic systems.

A critical innovation in mention-ranking architectures is higher-order inference, where span representations are iteratively updated based on their predicted antecedents. After an initial ranking pass, each span's embedding is refined by incorporating information from its top-ranked antecedent. This process repeats for multiple iterations, enabling transitive reasoning across coreference chains. For example, if mention A corefers with B, and B corefers with C, higher-order refinement allows the model to propagate information from C back to A, improving consistency across the entire coreference chain.

The canonical e2e-coref architecture jointly optimizes mention detection and coreference scoring in a single end-to-end model. Rather than treating mention detection as a separate preprocessing step, the model considers all possible spans up to a maximum length, assigns each a mention likelihood score, and prunes low-scoring spans. The surviving spans then participate in the antecedent ranking process. This joint optimization allows the model to learn mention boundaries that are optimal for coreference resolution, rather than relying on a separate mention detection system that may miss spans critical for building complete coreference chains.

Naively considering all possible spans and antecedents would be computationally intractable. Mention-ranking models employ two key pruning strategies:

• Span pruning: After computing mention scores, only the top-k spans (typically 0.4 * document length) are retained as candidate mentions
• Antecedent pruning: For each mention, only the nearest k preceding spans are considered as candidate antecedents, based on the linguistic observation that most coreference relationships are local These heuristics reduce the quadratic complexity of pairwise scoring to a manageable linear pass over the document while maintaining high recall on standard benchmarks like CoNLL-2012.

The final ranking decision uses a softmax over all candidate antecedent scores plus a dummy epsilon score for the 'no antecedent' case. This formulation elegantly handles singleton entities—mentions that do not corefer with any previous span—by allowing the model to assign them to the null antecedent. The probability distribution over candidates ensures that the model makes a globally coherent decision for each mention rather than applying an arbitrary threshold. This probabilistic formulation also enables the model to express uncertainty when multiple antecedents are plausible, which is common in ambiguous pronominal resolution scenarios.

The predecessor architecture that classifies each candidate antecedent-mention pair independently. Unlike mention-ranking, it makes a binary coreference decision for each pair without comparing all candidates simultaneously. This can lead to contradictory clustering where a pronoun is linked to two different antecedents in separate pairwise decisions. Mention-ranking resolves this by selecting a single, highest-scoring antecedent from the candidate set.

The scoring mechanism typically used inside a mention-ranking model to compute the compatibility between a mention and a candidate antecedent. It uses a learned bilinear transformation:

• Takes the vector representation of the mention and the antecedent
• Passes them through a feedforward network
• Computes a scalar score via a bilinear form This produces a single number representing how likely the antecedent is the correct coreferent.

An iterative refinement technique that allows mention-ranking models to perform transitive reasoning across coreference chains. After the first ranking pass, span representations are updated by attending over their predicted antecedents' representations. The model then re-ranks candidates using these enriched representations. This enables the model to learn that if mention A corefers with B, and B corefers with C, then A and C are also coreferent.

A computational efficiency technique that restricts the candidate search space before ranking. Common heuristics include:

• Distance pruning: Only consider antecedents within a fixed window
• Agreement constraints: Filter out candidates with mismatched number, gender, or animacy
• Syntactic constraints: Remove candidates that violate binding theory principles This reduces the quadratic complexity of pairwise scoring to near-linear in practice.

The fixed-length vector encoding a contiguous token sequence that serves as input to the mention-ranking scorer. In the canonical e2e-coref model, span representations are computed by concatenating:

• The hidden states of the span's start and end tokens from a bidirectional LSTM or SpanBERT
• A learned attention-weighted sum over all tokens in the span
• An embedding of the span's width This vector must capture both the mention's identity and its syntactic role.

The standard evaluation dataset for end-to-end coreference resolution, derived from OntoNotes 5.0. It contains documents across multiple genres including newswire, broadcast conversation, and web text. Models are evaluated using MUC, B³, and CEAF metrics, with the average F1 score (CoNLL F1) serving as the primary benchmark. Mention-ranking architectures have dominated this leaderboard since the introduction of neural coreference.