Cross-document Coreference

Unlike within-document resolution, CDC operates on a corpus-wide scale to merge mentions across thousands of documents. The core mechanism is a clustering algorithm (e.g., hierarchical agglomerative clustering or Chinese Whispers) that groups mentions based on pairwise similarity scores.

• Input: All extracted mentions from a document collection.
• Output: A set of clusters, where each cluster represents a unique real-world entity.
• Challenge: Scaling O(n²) pairwise comparisons to millions of mentions requires blocking and approximate nearest neighbor techniques.

CDC must resolve the same surface form to different entities based on document context. The word 'Jordan' might refer to the country in a geopolitics article, the basketball player in a sports report, and a river in a geography paper.

• Uses document-level topic vectors to weight entity candidates.
• Leverages temporal signals (publication dates) to constrain entity lifespan.
• Applies entity linking as a prerequisite step to normalize mentions to a knowledge base ID before cross-document clustering.

CDC systems aggregate evidence from heterogeneous sources to build a unified entity profile. A person entity might be referenced by name in one document, by pronoun in another, and by a Twitter handle in a third.

• Name variants: 'Barack Obama', 'President Obama', 'Obama'.
• Attributes: Aggregated properties like birth dates, job titles, and affiliations.
• Relations: Connections to other entities that provide structural constraints for disambiguation.
• This fusion process is the foundation of Knowledge Base Population (KBP).

A critical CDC function is singulation—ensuring that two distinct entities with the same name are not incorrectly merged. The system must also handle nil entities that have no corresponding entry in the target knowledge base.

• Singulation: Uses biographical attributes (e.g., different birth dates for 'John Smith') to split clusters.
• Nil prediction: Identifies mentions that refer to entities absent from the KB, preventing false linkage.
• Confidence scoring: Each cluster assignment carries a probability, allowing downstream systems to threshold based on precision/recall requirements.

Production CDC systems often operate on real-time document streams rather than static corpora. New documents require incremental cluster updates without full re-computation.

• Online clustering: Algorithms like threshold-based nearest-neighbor that assign incoming mentions to existing clusters or create new ones.
• Cluster splitting: A previously merged cluster may need to be divided as new, disambiguating evidence arrives.
• Temporal decay: Older, less-confident clusters may be pruned to manage storage and computational complexity.

CDC is evaluated using standard coreference metrics adapted for the cross-document scale, primarily B³ F1, CEAF, and MUC. The key metric is Cluster F1, which measures the harmonic mean of precision and recall over entity clusters.

• B³ (Bagga & Baldwin): Computes precision and recall for each mention, then averages.
• CEAF (Constrained Entity-Alignment F-Measure): Aligns predicted clusters to gold clusters using a similarity metric.
• LEA (Link-based Entity-Aware): Designed to resolve the metric bias against large entities present in MUC and B³.

The foundational within-document task that cross-document coreference extends. It identifies all expressions—pronouns, definite noun phrases, and named mentions—that refer to the same entity in a single text. For example, linking 'she', 'the CEO', and 'Jane Smith' to the same individual. Modern systems use span-based neural architectures that score antecedent-mention pairs. Without robust within-document resolution, cross-document clustering lacks coherent mention chains to connect.

The statistical discipline of identifying records across structured databases that refer to the same real-world entity when no shared unique key exists. Unlike coreference, which operates on unstructured text, record linkage works on tabular data with defined attributes. The Fellegi-Sunter model provides the probabilistic framework, calculating match weights based on attribute agreement patterns. Key techniques include:

• Blocking to reduce quadratic comparison complexity
• Fuzzy string matching for typo-tolerant field comparison
• Expectation-Maximization for unsupervised parameter estimation

The automated pipeline that ingests unstructured text, extracts entities and relations, and inserts them into a structured knowledge base. Cross-document coreference is the clustering backbone of KBP—it ensures that facts extracted from different articles about the same person are merged under a single entity node. The TAC KBP benchmark evaluates systems on:

• Entity discovery and linking
• Slot filling for attributes like birth dates
• Event and sentiment tracking across documents

The enterprise discipline of creating a 360-degree customer view by merging identity fragments from CRM, transactional, and web interaction systems. It applies cross-document coreference principles to structured and semi-structured data. Modern implementations use graph-based entity resolution where nodes represent records and edges represent match probabilities. This enables:

• Fraud detection by linking synthetic identities
• Personalization by unifying browsing and purchase history
• Compliance with GDPR right-to-access requests

A specialized variant that clusters mentions of the same real-world occurrence across documents, distinguishing between event instances and generic classes. For example, determining that 'the explosion at the Beirut port' in one article and 'Tuesday's devastating blast in Lebanon' in another refer to the same event. This requires modeling:

• Spatiotemporal arguments (where and when)
• Participant overlap (who was involved)
• Event subtyping hierarchies for granular classification