Mention Pair Model

The core mechanism reduces coreference to a binary decision: for any two mentions (i, j), the model independently predicts whether they are coreferent. This is implemented by feeding a concatenated vector representation of both spans through a feedforward network with a softmax output layer. The model scores each candidate antecedent-mention pair in isolation, without considering global coherence or other links in the chain.

Traditional mention-pair models rely heavily on hand-crafted linguistic features to represent the relationship between two spans. These include:

• Lexical features: String match, head word match, edit distance
• Syntactic features: Binding theory constraints, i-commands, c-commands
• Distance features: Token distance, sentence distance, mention count distance
• Agreement features: Number, gender, and animacy compatibility
• Semantic class features: WordNet hypernyms, named entity type compatibility

Before pairwise scoring, the model generates a set of candidate antecedents for each mention. This is typically done using a deterministic sieve that applies high-precision filters:

• Restricting candidates to mentions within a fixed window of preceding sentences
• Filtering by agreement constraints (number, gender, animacy)
• Applying syntactic filters based on binding theory principles This pruning step is critical for computational tractability, as exhaustive pairwise comparison scales quadratically with document length.

The model is trained on labeled coreference chains by generating training instances from all possible mention pairs. For each anaphoric mention, the correct antecedent forms a positive instance, while all non-coreferent preceding mentions form negative instances. This creates a highly imbalanced dataset where negative pairs vastly outnumber positive ones, requiring careful sampling strategies or weighted loss functions during optimization.

A fundamental limitation of the mention-pair architecture is that independent pairwise decisions do not guarantee transitive closure. The model may predict that mention A is coreferent with B, and B is coreferent with C, but A is not coreferent with C. This violates the equivalence relation property of coreference. Post-processing heuristics, such as agglomerative clustering over pairwise scores, are required to enforce consistent chain formation.

The mention-pair architecture directly led to the development of mention-ranking models, which address key limitations. Rather than making independent binary decisions, mention-ranking models score all candidate antecedents for a given mention and select the highest-scoring one via a softmax over candidates. This implicitly captures competition among antecedents and eliminates the need for a binary classification threshold. Modern neural systems like e2e-coref use this ranking formulation with learned span representations.

The dominant neural architecture that replaced independent pairwise scoring. Instead of classifying each pair in isolation, it scores all candidate antecedents for a given mention and selects the highest-ranked one. This approach naturally handles the single-antecedent constraint and allows the model to compare candidates against each other. Modern implementations use biaffine attention to compute pairwise scores efficiently, with the final coreference chains built by greedily linking each mention to its highest-scoring antecedent.

An iterative refinement technique that addresses a key limitation of the Mention Pair Model: the inability to reason transitively across chains. After an initial pass of antecedent scoring, span representations are updated based on the representations of their predicted antecedents. This allows information to propagate through the coreference chain, enabling the model to implicitly learn that if A corefers with B, and B corefers with C, then A should corefer with C. Typically applied for 2-3 iterations before diminishing returns set in.

A computational efficiency technique critical for making mention pair models tractable. Without pruning, a document with n mentions would require evaluating O(n²) pairs. Antecedent pruning restricts the candidate search space by applying heuristic filters:

• Distance pruning: Only consider antecedents within a fixed window
• Syntactic constraints: Filter based on binding theory violations
• Agreement features: Remove candidates with mismatched number or gender
• String matching: Prioritize exact string matches as high-confidence candidates Modern systems typically keep 50-250 candidates per mention.

A scoring mechanism that computes a pairwise score between mention and antecedent representations using a learned bilinear transformation. Given a mention representation m and a candidate antecedent representation a, the score is computed as m^T W a + U[m; a] + b, where W is a learned weight matrix. This allows the model to capture complex interactions between the two spans while remaining computationally efficient. Biaffine attention has become the standard scoring function in mention-ranking architectures, replacing simpler feedforward networks used in earlier mention pair models.