Deep Biaffine Parser

The core innovation of the architecture is the use of a deep biaffine classifier to score dependency arcs. Unlike simpler dot-product or single-layer affine transformations, the biaffine layer applies a bilinear transformation to distinct representations of the head and dependent. This means the model learns an affine transformation in a lower-dimensional space before computing the score, allowing it to capture asymmetric relationships where the features that make a word a good head differ from those that make it a good dependent. The operation is: s = h_head^T * U * h_dep + W * (h_head ⊕ h_dep) + b, where U is a learned weight tensor.

The parser does not use a single vector to represent a word for all tasks. Instead, it creates four distinct, specialized representations from the shared BiLSTM hidden states using separate multi-layer perceptrons (MLPs):

• Head representation for arc existence prediction
• Dependent representation for arc existence prediction
• Head representation for relation label classification
• Dependent representation for relation label classification This factorization allows the model to learn task-specific features, significantly improving both attachment and labeling accuracy.

As a graph-based parser, the model scores all possible arcs in a sentence simultaneously to construct a complete weighted directed graph. The final parse tree is found by applying the Chu-Liu/Edmonds algorithm to extract the maximum spanning tree. This global approach contrasts with transition-based methods that make local, greedy decisions. By considering the entire sentence structure at once, the deep biaffine parser mitigates error propagation and handles non-projective dependencies naturally, making it robust for languages with free word order.

The architecture cleanly separates two subtasks:

• Arc Prediction: A biaffine classifier scores the likelihood of a directed edge existing between every pair of words. The root node is treated as a special dummy token with its own learned representation.
• Label Classification: For every predicted arc, a second biaffine classifier assigns a dependency relation label (e.g., nsubj, dobj, amod) from the Universal Dependencies inventory. This two-stage design allows the model to first determine the unlabeled tree structure before classifying the semantic nature of each grammatical relation.

The foundation of the parser is a multi-layer bidirectional LSTM that processes the sequence of input word embeddings and part-of-speech tag embeddings. The BiLSTM captures long-range syntactic context, producing hidden states that encode both left and right sentential context for each token position. These contextualized representations are then fed into the factorized MLPs. The use of recurrent layers ensures that the biaffine scorer operates on representations that are aware of the entire sentence, not just local n-gram windows.

The original implementation applies variational dropout aggressively throughout the network—on the input embeddings, the BiLSTM hidden states, and the MLP transformations. This regularization technique prevents co-adaptation of neurons and forces the model to learn redundant representations. The result is a parser that generalizes well to unseen syntactic constructions and domains without requiring extensive feature engineering or external resources beyond word vectors and POS tags.

The overarching paradigm in which the Deep Biaffine Parser operates. Unlike transition-based methods, graph-based parsing scores all possible arcs in a sentence simultaneously to find the highest-scoring maximum spanning tree.

• Global Optimization: Finds the best tree structure for the whole sentence.
• Algorithm: Typically uses the Chu-Liu/Edmonds algorithm for decoding.
• Advantage: Mitigates error propagation common in greedy, left-to-right decisions.

The core neural mechanism that gives the parser its name. It applies a bilinear transformation to compute scores for all possible head-dependent pairs.

• Function: A shallow, trainable function that replaces traditional MLP-based scoring.
• Efficiency: Computes an n x n matrix of arc scores in a single operation.
• Dual Use: Applied separately to predict both the syntactic head and the dependency label for each token.

The contextualized input layer for the biaffine classifier. A Bidirectional Long Short-Term Memory network processes the sequence of word vectors to capture infinite context on both sides.

• Contextualization: Each token representation is conditioned on the entire sentence.
• Input: Typically fed by concatenated fastText or GloVe embeddings and character-level CNNs.
• Output: Hidden states that serve as the distinct feature vectors for heads and dependents.

The primary evaluation metric for dependency parsers, measuring the percentage of tokens assigned both the correct syntactic head and the correct dependency relation label.

• Calculation: (Number of tokens with correct head and label) / (Total number of tokens).
• Significance: A strict metric that penalizes both structural and labeling errors.
• Benchmark: The Deep Biaffine Parser achieved state-of-the-art LAS on the Penn Treebank upon release.

A cross-linguistically consistent framework for grammatical annotation that defines a universal set of part-of-speech tags and dependency relations. The Deep Biaffine Parser is a standard baseline for UD treebanks.

• Goal: Facilitate multilingual parser development.
• Format: Annotations are stored in the CoNLL-U format.
• Relations: Uses a fixed inventory of universal relations like nsubj, dobj, and amod.

A first-order parsing assumption where the score of a dependency tree is the sum of the scores of its individual arcs, assuming independence between edges.

• Tractability: Makes global decoding computationally feasible.
• Limitation: Cannot capture interactions between sibling or grandparent arcs.
• Contrast: Higher-order parsing extends this to include features from multiple arcs, but the biaffine model achieves high accuracy even with first-order factorization.