Biaffine Classifier

The core mechanism applies a learned bilinear transformation to pairs of input vectors. Unlike a simple dot product or linear layer, the biaffine operation uses a learned weight matrix to capture asymmetric relationships between the start and end representations of a potential entity span. The operation computes score = x_start^T * W * x_end + U * [x_start; x_end] + b, where W models the interaction between the two vectors, U projects their concatenation, and b is a bias term. This allows the model to learn that certain start token representations are highly compatible with specific end token representations, directly encoding the structural prior that entities are contiguous spans.

Biaffine classifiers enable direct span enumeration by scoring all possible start-end token pairs in a sentence. For a sequence of length n, the model produces an n x n x c scoring tensor, where c is the number of entity types. This eliminates the need for Viterbi decoding or linear-chain CRFs, as the model independently scores each span rather than modeling global label sequence dependencies. The approach naturally handles nested NER scenarios where entities overlap, such as a person name embedded within an organization mention. During inference, spans exceeding a threshold score are selected, and non-maximum suppression resolves conflicting predictions.

Before the biaffine operation, two separate feedforward neural networks project the shared contextualized token representations into distinct start and end vector spaces. This is critical because the same token embedding from a model like BERT encodes different information depending on whether the token serves as a span start or span end. The start projection learns to identify tokens that initiate entities, while the end projection learns to recognize tokens that terminate them. These projections typically use a ReLU activation and dropout, creating an information bottleneck that forces the model to extract span-relevant features from the rich but general-purpose contextual embeddings.

The biaffine classifier incorporates a multi-class classification component that simultaneously predicts the entity type of each scored span. The weight matrix W is typically a tensor of shape d x d x c, where d is the hidden dimension and c is the number of entity classes. This means each entity type learns its own bilinear interaction pattern between start and end vectors. For example, the model can learn that PER entities have a distinct start-end interaction signature compared to LOC entities. The type-specific scoring enables the model to assign different confidence scores to the same span for different entity types, supporting fine-grained entity typing within a unified architecture.

Training uses a binary cross-entropy loss applied independently to each possible span in the sentence. The ground truth is a sparse matrix where valid entity spans are labeled as positive examples and all other spans are negative. This formulation treats NER as a set prediction problem rather than a sequence labeling problem. The loss function heavily penalizes false positives and false negatives at the span level. To handle the extreme class imbalance between the few true entities and the vast number of non-entity spans, techniques like focal loss or negative sampling are often employed to prevent the model from simply predicting all spans as negative.

Biaffine classifiers are typically stacked on top of transformer-based encoders like BERT, RoBERTa, or SpanBERT. The encoder produces contextualized token representations that capture long-range dependencies and semantic information. The biaffine layer then leverages these rich representations for span scoring. This architecture is end-to-end differentiable, allowing gradients from the span-level loss to flow back through the biaffine operation into the encoder during fine-tuning. The combination of a powerful pre-trained encoder with a structurally appropriate biaffine decoder consistently achieves state-of-the-art results on benchmarks like CoNLL-2003 and OntoNotes 5.0, particularly for nested and discontinuous entity recognition tasks.