BERT-based SRL

Unlike static word embeddings, BERT generates a dynamic representation for each predicate token that is conditioned on the entire sentence context. This allows the model to disambiguate polysemous verbs (e.g., 'run' in 'run a company' vs. 'run a marathon') before argument classification begins. The final hidden state corresponding to the predicate token serves as a rich, context-aware feature vector for downstream role labeling.

BERT-based SRL models typically operate directly on raw text, eliminating the pipeline dependency on external syntactic parsers. The architecture enumerates all possible text spans up to a maximum length and computes a score for each span-role pair using a feedforward network over BERT embeddings. This span-based approach mitigates error propagation from syntactic parsing, a critical weakness of earlier SRL systems.

The core innovation of BERT—bidirectional self-attention—allows every token to attend to every other token simultaneously. For SRL, this means an argument candidate can condition its representation on both the predicate and other potential arguments before role assignment. This global context helps resolve complex phenomena like coreferential arguments and long-distance dependencies that challenge unidirectional models.

Advanced BERT-based SRL architectures jointly model predicate identification, predicate disambiguation, and argument classification in a unified framework. By sharing the same contextualized representations, the model learns interactions between these sub-tasks. For example, knowing that 'book' is a verb (predicate) rather than a noun helps the model correctly identify 'the flight' as the Theme argument.

BERT's pre-training objectives—Masked Language Modeling (MLM) and Next Sentence Prediction (NSP)—implicitly teach syntactic and semantic knowledge. Fine-tuning on a relatively small annotated corpus like OntoNotes 5.0 transfers this general linguistic competence to SRL. This drastically reduces the need for task-specific feature engineering and achieves state-of-the-art F1 scores exceeding 87% on in-domain test sets.

Many BERT-based SRL architectures employ a deep biaffine attention layer between predicate and argument representations. This mechanism computes a pairwise score using a learned bilinear transformation, explicitly modeling the asymmetric relationship between a predicate and its potential arguments. The biaffine scorer captures which specific role a span fills relative to a given predicate, outperforming simpler concatenation-based classifiers.

The parent task that BERT-based models solve. SRL identifies the predicate-argument structure of a sentence, answering 'who did what to whom, when, where, and how.' It assigns labels like Agent, Patient, and Instrument to constituents relative to a target verb.

• Transforms unstructured text into structured relational data
• Critical for downstream tasks like information extraction and question answering
• BERT-based approaches treat this as a sequence tagging or span classification problem

A neural architecture paradigm that enumerates and scores arbitrary text spans as potential arguments, removing the dependency on pre-computed syntactic parse trees. Unlike BIO tagging, span-based methods directly model the start and end boundaries of arguments.

• Uses biaffine attention to score predicate-argument pairs
• Eliminates error propagation from external parsers
• Enables joint modeling of argument identification and classification

A corpus annotated with predicate-argument relations and verb-specific semantic roles, serving as the foundational training resource for supervised SRL systems. PropBank defines framesets for each verb sense, specifying numbered arguments (Arg0, Arg1, Arg2) with sense-specific definitions.

• Arg0 typically maps to the Agent or causer
• Arg1 maps to the Patient or theme
• ArgM labels denote modifiers like temporal, locative, and manner

The landmark benchmarking challenge based on the OntoNotes corpus that drove significant progress in end-to-end SRL systems. It evaluates models on joint coreference resolution and semantic role labeling across multiple genres including newswire, broadcast, and web text.

• Standard train/dev/test split used in most SRL papers
• BERT-based models achieved state-of-the-art F1 scores exceeding 87%
• Evaluates both span identification and role classification accuracy

A deep learning scoring mechanism that computes pairwise scores between a predicate representation and its potential argument spans using a low-rank bilinear transformation. This mechanism is central to modern SRL architectures.

• Computes scores as: s(i,j) = h_i^T U h_j + W[h_i; h_j] + b
• Enables efficient scoring of all possible predicate-argument pairs
• Jointly predicts argument span boundaries and semantic role labels

A heuristic or learned filtering step that reduces the search space for potential arguments by discarding constituents highly unlikely to be associated with a given predicate. Critical for computational efficiency in span-based models.

• Reduces candidate arguments from O(n²) to a manageable subset
• Can be rule-based (e.g., removing punctuation) or learned via a binary classifier
• BERT's contextualized embeddings often make pruning more accurate by encoding syntactic distance