Global Pointer

Unlike standard additive attention, Global Pointer uses multiplicative attention to compute span scores. For a sequence of length n, it produces an n×n matrix where each cell (i, j) represents the score of a span from token i to token j.

• Score calculation: q_i^T * k_j, where q_i and k_j are linear projections of token representations
• Asymmetric design: Separate projections for start (query) and end (key) positions
• Global view: Every possible span is scored simultaneously in a single forward pass
• No length constraints: Handles spans of arbitrary length without beam search or dynamic programming

Global Pointer eliminates the Conditional Random Field (CRF) layer traditionally used in sequence labeling. Instead of modeling label transitions between adjacent tokens, it directly classifies span candidates.

• Independent span scoring: Each span is scored independently without transition constraints
• Simplified training: Standard binary cross-entropy loss replaces CRF's log-likelihood optimization
• No Viterbi decoding: Inference is a simple threshold on span scores, avoiding dynamic programming
• Parallelizable: All spans are evaluated simultaneously, unlike sequential CRF decoding

Global Pointer integrates Rotary Position Embeddings to encode relative positional information directly into the attention scores, enabling the model to understand token distances without absolute position encodings.

• Relative position awareness: Encodes the distance between start and end tokens naturally
• Decay with distance: Longer spans receive naturally attenuated scores
• Compatibility: Works seamlessly with transformer architectures that use RoPE
• Theoretical grounding: Based on rotating query and key vectors by position-dependent angles

Global Pointer natively supports nested NER by predicting multiple entity types as separate channels. Each entity type gets its own scoring matrix, allowing overlapping spans of different types to coexist.

• Type-specific matrices: One n×n matrix per entity type, enabling independent scoring
• Hierarchical recognition: A 'Bank of America' mention can simultaneously be tagged as Organization and Location
• No conflict resolution: Overlapping spans of different types don't compete
• Multi-label output: Each span can receive multiple type labels without mutual exclusion

During inference, Global Pointer applies learnable or calibrated thresholds per entity type to convert raw span scores into binary predictions. This replaces the global argmax of token-level classifiers.

• Per-type calibration: Each entity type can have its own optimal threshold tuned on validation data
• Precision-recall tradeoff: Thresholds can be adjusted post-hoc without retraining
• Confidence filtering: Low-scoring spans are filtered out, reducing false positives
• No label dependency: Unlike CRF, thresholding doesn't consider neighboring label transitions

While both Global Pointer and Biaffine Classifiers score start-end pairs, they differ fundamentally in their scoring functions and architectural assumptions.

• Global Pointer: Uses q_i^T * k_j with separate linear projections
• Biaffine: Uses s_i^T * U * e_j + w^T * [s_i; e_j], introducing a bilinear interaction matrix U
• Parameter efficiency: Global Pointer avoids the large interaction matrix, reducing parameters
• Empirical parity: Both achieve comparable state-of-the-art results on benchmarks like CoNLL-2003 and OntoNotes

A discriminative probabilistic graphical model for structured prediction. In traditional NER, a linear-chain CRF sits atop token-level emissions to model the conditional probability of a label sequence given an observation sequence. It learns transition constraints—such as preventing an I-ORG tag from following a B-PER tag—and uses Viterbi decoding to find the optimal global label path. Global Pointer architectures often remove the CRF entirely.

A token-level annotation scheme where each token is tagged as Beginning, Inside, or Outside of a named entity span. This is the standard encoding for sequence labeling models like BERT-NER. Its primary limitation is the inability to represent nested entities—where one entity is contained within another—without complex multi-label extensions. Span-based methods like Global Pointer bypass this constraint.

The task of recognizing entities that are hierarchically embedded within other entities. For example, in 'the University of California, Berkeley campus', both 'University of California' (Organization) and 'Berkeley' (Location) are valid entities. Traditional BIO tagging fails here because a token cannot have two labels. Global Pointer naturally handles nesting by independently scoring all start-end pairs.

A dynamic programming algorithm that finds the most probable sequence of hidden states in a linear-chain CRF. It efficiently computes the maximum over all possible label paths by recursively selecting the optimal predecessor state at each step. This ensures globally coherent entity predictions. Span-based decoders like Global Pointer replace this with a simpler threshold-based selection over scored spans.