Document-Level Relation Extraction (DocRED)

Unlike traditional RE, DocRED requires synthesizing evidence that is distributed across multiple sentences. An entity pair may have a relationship implied by facts stated in different paragraphs, demanding that models aggregate non-local contextual clues rather than relying on a single syntactic window.

• Example: Sentence 1 states 'John works at Google.' Sentence 5 states 'Google is headquartered in Mountain View.' The model must infer the cross-sentence relation (John, works_in, Mountain View).
• This necessitates architectures with long-range attention mechanisms beyond standard transformer windows.

A single document-level entity is often referenced by multiple distinct mentions (e.g., 'Apple', 'the tech giant', 'it', 'the Cupertino company'). DocRED models must perform implicit coreference resolution to merge these disparate mentions into a unified entity representation before classifying relations.

• Challenge: A relation may be stated using a pronoun ('she founded it') that requires resolving 'she' and 'it' to their canonical entities.
• This demands mention-pooling strategies that aggregate contextualized embeddings from all occurrences of an entity.

DocRED often requires models to perform multi-hop logical inference to deduce implicit relationships. The target relation is not explicitly stated but must be derived by combining two or more explicitly stated facts.

• Induction: If (X, parent_of, Y) and (Y, parent_of, Z) are stated, the model must infer (X, grandparent_of, Z).
• Deduction: Combining a spatial relation with a located-in relation to infer a new spatial containment.
• This pushes models beyond pattern matching toward primitive symbolic reasoning over extracted triples.

The DocRED benchmark provides 101,873 documents annotated with 132,375 entities and 56,354 relational facts across 96 relation types derived from Wikipedia and Wikidata. This scale enables training complex neural models.

• Evidence Annotation: Crucially, DocRED annotates the specific sentences supporting each relation, enabling the development of evidence-aware models that must locate supporting text.
• Train/Dev/Test Split: 3,053 docs for training, 1,000 for dev, and 1,000 for test, providing a standardized evaluation protocol.

DocRED uniquely pairs relation extraction with an evidence extraction sub-task. A predicted relation is only considered correct if the model also identifies at least one supporting sentence from the document.

• Joint Evaluation: Metrics like Ign F1 and F1 evaluate relation extraction performance, while Evi F1 specifically measures the model's ability to locate supporting evidence.
• This dual requirement ensures models are not exploiting spurious correlations and are genuinely reading and grounding their predictions in the text.

The dominant approach to DocRED involves constructing a heterogeneous document graph where nodes represent entities, mentions, and sentences, and edges represent co-occurrence, coreference, and discourse relations.

• Graph Neural Networks (GNNs) or Graph Attention Networks (GATs) then propagate information across this structure to model inter-entity dependencies.
• Transformer-based alternatives use special entity markers and hierarchical attention to achieve similar cross-sentence information flow without explicit graph construction.

A semantic relationship where the head entity and tail entity appear in different sentences within the same document. This is the core challenge that distinguishes DocRED from sentence-level RE.

• Requires discourse-level reasoning to connect mentions across paragraph boundaries
• Often involves coreference resolution to link pronouns to named entities
• Example: 'Alice joined Google in 2019. She now leads the AI division.' — the 'employed_by' relation spans two sentences

The task of identifying all expressions that refer to the same real-world entity. Essential preprocessing for DocRED because entities are frequently mentioned via pronouns, aliases, or nominal phrases across a document.

• Links 'Apple Inc.' → 'the company' → 'it' → 'the Cupertino giant'
• Without coreference resolution, cross-sentence relations remain invisible to the model
• Modern approaches use span-based neural models that score antecedent-mention pairs

GNNs model a document as a heterogeneous graph where nodes represent entities, mentions, and sentences, while edges encode syntactic, semantic, and discourse connections.

• Message passing propagates information between distant entity mentions
• Captures multi-hop reasoning paths: Entity A → Sentence 1 → Entity B → Sentence 3 → Entity C
• Variants include GAT (Graph Attention Networks) and R-GCN (Relational GCN) for edge-type awareness

A method for automatically generating training data by aligning a knowledge base with a text corpus. If a KB contains a relation between two entities, all sentences mentioning both are labeled with that relation.

• Eliminates costly manual annotation for DocRED-scale datasets
• Introduces noisy labels — not every co-occurrence expresses the KB relation
• Multi-instance learning mitigates noise by aggregating signals across all mention pairs in a document

A modeling paradigm that simultaneously identifies entities and the relationships between them in a single unified step, rather than as a pipeline.

• Prevents error propagation where entity extraction mistakes cascade into relation classification
• Uses shared representations that allow entity recognition to inform relation prediction and vice versa
• Common architectures include table-filling approaches and sequence-to-graph transformers

The process of adjusting a model's predicted probability to reflect the true likelihood of correctness. Critical for DocRED because distant supervision introduces label noise.

• A well-calibrated model outputs 0.8 probability when it is correct 80% of the time
• Techniques: temperature scaling, Platt scaling, and isotonic regression
• Enables reliable thresholding for knowledge graph population pipelines