Neural SRL

Neural SRL models learn contextualized representations directly from raw text without relying on hand-crafted features or external syntactic parsers. Using architectures like BiLSTMs and Transformers, these systems automatically discover the linguistic patterns that signal predicate-argument relationships.

• Eliminates manual feature engineering pipelines
• Learns task-specific representations from pretrained embeddings
• Captures long-range dependencies between predicates and arguments
• Adapts to new domains without redesigning feature templates

Modern span-based architectures enumerate candidate argument spans and score them using biaffine attention mechanisms, removing the dependency on pre-computed parse trees entirely.

Fine-tuning pretrained BERT and its variants has become the dominant paradigm for neural SRL. The model processes the sentence with a special predicate marker token, generating contextualized embeddings that encode both syntactic and semantic information simultaneously.

• Predicate-aware attention patterns emerge during fine-tuning
• Multi-head self-attention captures argument-predicate relationships
• Single model handles both argument identification and classification
• Transfer learning from masked language modeling improves generalization

SpanBERT and RoBERTa variants achieve particularly strong results by better representing contiguous text spans, which align naturally with SRL argument boundaries.

Neural SRL splits into two architectural paradigms. Span-based SRL treats arguments as arbitrary contiguous text spans, scoring them independently of syntactic structure. Dependency-based SRL operates on dependency trees, predicting semantic roles for syntactic dependents of the predicate.

• Span-based: Enumerates all possible spans, uses biaffine scoring
• Dependency-based: Leverages syntactic head-dependent relations as structural priors
• Span methods excel when parse quality is unreliable
• Dependency methods benefit from explicit syntactic constraints

Biaffine attention computes pairwise scores between predicate and argument representations using a low-rank bilinear transformation, enabling efficient scoring over large candidate sets.

Neural SRL can be reformulated as a token-level sequence labeling task using the BIO (Beginning-Inside-Outside) tagging scheme. Each token receives a tag indicating whether it begins an argument, continues inside one, or falls outside any argument for a given predicate.

• Transforms structured prediction into token classification
• Enables use of standard sequence models like BiLSTM-CRF
• B-tag: First token of an argument span
• I-tag: Continuation tokens within the same argument
• O-tag: Tokens outside any argument

This formulation simplifies the architecture but can struggle with discontinuous arguments and overlapping roles, which span-based methods handle more naturally.

Syntax-aware neural SRL jointly optimizes syntactic parsing and semantic role labeling objectives, allowing the shared encoder to learn representations that benefit both tasks. This approach explicitly injects syntactic knowledge as a structural prior rather than relying on external parsers.

• Shared BiLSTM or Transformer encoder between tasks
• Syntactic supervision improves argument boundary detection
• Multi-task learning reduces overfitting on small SRL datasets
• Dependency arc predictions guide argument attachment decisions

Joint models achieve competitive performance while producing both syntactic and semantic analyses in a single forward pass, reducing computational overhead in production pipelines.

Argument pruning reduces the candidate search space by filtering out constituents that are highly unlikely to serve as arguments for a given predicate. Neural pruners learn to score candidate spans and retain only the top-k most promising options before the expensive role classification step.

• Reduces computational cost from quadratic to near-linear
• Learned pruning models achieve >95% recall on gold arguments
• Critical for scaling span-based SRL to long documents
• Uses lightweight scoring networks for fast filtering

Without pruning, span-based enumeration over long sentences becomes computationally prohibitive, making this step essential for practical deployment.

A modern neural architecture that eliminates the dependency on pre-computed syntactic parse trees. Instead of relying on syntactic constituents, the model enumerates and scores arbitrary text spans as potential arguments for a given predicate.

• Directly models all possible spans up to a maximum width
• Uses biaffine attention to score predicate-span pairs
• Achieves state-of-the-art results without pipeline errors from syntactic parsers
• Example: In "John ate the apple," the model scores [the apple] as a high-probability span for the ARG1 role of "ate"

A high-performance approach that fine-tunes a pre-trained BERT model for semantic role labeling. The deep contextualized embeddings generated by BERT capture nuanced predicate senses and argument boundaries without explicit syntactic features.

• Replaces traditional feature engineering with contextualized word representations
• Typically formulated as a BIO tagging sequence labeling task
• Jointly encodes the predicate and sentence using special tokens (e.g., [SEP])
• Outperforms traditional feature-based and BiLSTM models on CoNLL-2012 benchmarks

A sequence labeling encoding that transforms SRL into a token-level classification problem. Each token is tagged as Beginning, Inside, or Outside of a semantic argument, with role-specific prefixes.

• Example: "B-ARG0" marks the first token of an Agent argument
• Enables simple linear-chain CRF or softmax decoding over token sequences
• Eliminates the need for span enumeration, reducing computational complexity
• Standard formulation for BERT-based SRL systems on the CoNLL-2005 and 2012 datasets

A deep learning scoring mechanism that computes pairwise scores between a predicate and its potential arguments using a low-rank bilinear transformation. Originally developed for dependency parsing, it is now a core component of span-based SRL architectures.

• Computes scores as: s(i,j) = h_i^T U h_j + W[h_i; h_j] + b
• The low-rank decomposition of U reduces parameter count and prevents overfitting
• Applied to score both span boundaries and predicate-argument relationships
• Enables efficient, parallelized scoring of all possible predicate-argument pairs

A landmark benchmarking challenge based on the OntoNotes corpus that drove significant progress in end-to-end semantic role labeling. The shared task evaluates systems on both argument identification and classification across multiple genres.

• Uses the OntoNotes 5.0 corpus with PropBank predicate-argument annotations
• Evaluates on newswire, broadcast, telephone, and web text genres
• Standard metrics: F1 score for argument identification and classification
• Catalyzed the transition from feature-based to neural SRL architectures

A modeling paradigm that explicitly incorporates syntactic parse tree information as a structural prior to improve semantic role predictions. While span-based models remove the hard dependency on syntax, syntax-aware models use syntactic features as soft inductive biases.

• Integrates dependency or constituency tree embeddings into the neural architecture
• Uses graph neural networks to propagate information along syntactic edges
• Improves performance on long-distance dependencies and rare predicate senses
• Demonstrates that syntax remains a useful auxiliary signal even in the deep learning era