Span Pruning

A learned mention scorer assigns a likelihood score $s_m(i)$ to every possible span up to a maximum width $L$. Spans with scores below a fixed threshold are discarded. This reduces the candidate pool from $O(n^2)$ to a manageable subset, typically keeping only the top $\lambda T$ spans where $T$ is the number of tokens and $\lambda$ is a hyperparameter (e.g., 0.4).

A hard linguistic constraint limits the maximum length of a candidate mention. By capping spans at a fixed number of tokens (commonly 10-30), the system eliminates arbitrarily long, semantically meaningless text segments. This is based on the observation that named entities and noun phrases rarely exceed a certain length in natural language.

Before any pairwise comparison, a feedforward network computes a unary representation for each span using its boundary tokens and an attention-weighted sum of its internal tokens. This representation encodes the span's inherent 'mention-worthiness' independent of any antecedent, enabling rapid, parallelizable filtering.

Without pruning, a document with $n$ tokens generates $O(n^2)$ candidate spans, and pairwise antecedent scoring costs $O(n^3)$. Span pruning reduces this to $O(n)$ candidates, making the subsequent mention-ranking model tractable for long documents. This is the primary enabler for end-to-end neural coreference on real-world text.

In the canonical e2e-coref architecture, span pruning is not a separate pipeline step but is learned jointly with coreference scoring. The mention scorer is trained end-to-end using the same loss function, allowing the model to learn which spans are useful for coreference chains rather than relying on a static, hand-crafted definition of a 'mention'.

A key advantage of the span-ranking approach is that pruning does not enforce non-overlapping boundaries. The system retains multiple overlapping spans (e.g., '[The CEO of [Microsoft]]') and allows the coreference scorer to determine which level of granularity is correct. Pruning only removes spans with low mention likelihood, not overlapping ones.

The prerequisite subtask that identifies all possible text spans referring to an entity. Span pruning directly filters the output of mention detection by removing spans with low mention likelihood scores before they enter the computationally expensive coreference scoring phase. A high-recall mention detector is essential, as spans pruned here are permanently lost.

A neural architecture that scores all candidate antecedents for a given mention and selects the highest-ranked one. Span pruning is the primary optimization for these models. By reducing the candidate mention set from all possible spans to a manageable subset, the antecedent scoring matrix shrinks dramatically, making end-to-end training feasible on long documents.

A fixed-length vector encoding a contiguous token sequence, typically computed from the hidden states of the span's start and end tokens using SpanBERT. The quality of a span's representation directly determines the accuracy of the pruning scorer. Poor representations lead to the erroneous pruning of valid mentions, introducing a hard ceiling on final coreference performance.

A complementary efficiency technique that restricts the candidate antecedent search space after mention detection. While span pruning filters which spans are considered valid mentions, antecedent pruning limits how far back the model looks for coreferring antecedents, often using a distance heuristic or syntactic constraints to maintain linear complexity.

An iterative refinement technique where span representations are updated based on their predicted antecedents, enabling transitive reasoning across chains. Efficient span pruning is a prerequisite for higher-order inference in production, as the iterative updates multiply the computational cost of an already quadratic scoring process.