Dependency Treebank

Every sentence is manually annotated by trained linguists following strict guidelines. This human-curated data provides the ground truth for supervised learning. Key aspects include:

• Inter-annotator agreement: Multiple annotators label the same sentences to measure consistency, with target agreement scores typically above 95% for attachment decisions.
• Adjudication: Disagreements are resolved by a senior linguist to create a single gold parse.
• Consistency: Annotation manuals define exactly how specific constructions (relative clauses, coordination, prepositional phrases) should be treated.

Each arc in the treebank carries a grammatical relation label from a standardized inventory. These labels capture the syntactic function of the dependent relative to its head:

• Core arguments: nsubj (nominal subject), obj (direct object), iobj (indirect object)
• Modifiers: amod (adjectival modifier), advmod (adverbial modifier), nummod (numeric modifier)
• Function words: det (determiner), case (case marking), aux (auxiliary verb) The Universal Dependencies framework defines 37 universal relation types applicable across languages.

Treebanks are distributed in the CoNLL-U format, a tab-separated text standard. Each token occupies one line with 10 fields:

• ID: Token index (integer or range for multiword tokens)
• FORM: The surface form as it appears in the sentence
• LEMMA: Base or dictionary form
• UPOS: Universal part-of-speech tag
• XPOS: Language-specific POS tag (optional)
• FEATS: Morphological features (e.g., Number=Sing|Person=3)
• HEAD: Index of the syntactic head (0 for root)
• DEPREL: Dependency relation label to the head
• DEPS: Enhanced dependency graph (optional)
• MISC: Any other annotation (e.g., SpaceAfter=No)

Modern treebanks follow the Universal Dependencies (UD) framework, which defines a consistent annotation scheme across over 100 languages. This enables:

• Multilingual parser training: A single architecture can be trained on multiple treebanks simultaneously.
• Cross-lingual transfer: A parser trained on a high-resource language like English can be applied to a low-resource language using multilingual embeddings.
• Typological research: Linguists can compare syntactic structures across language families using a uniform representation. The UD project currently includes over 200 treebanks spanning diverse language families.

Treebanks are split into training, development, and test sets to enable rigorous parser evaluation. The primary metrics are:

• Labeled Attachment Score (LAS): The percentage of tokens assigned both the correct head and the correct dependency relation label. This is the most stringent metric.
• Unlabeled Attachment Score (UAS): The percentage of tokens assigned the correct head, ignoring relation labels.
• Label Accuracy Score (LA): The percentage of tokens with the correct relation label, given the correct head. State-of-the-art neural parsers achieve LAS scores above 95% on English treebanks.

Beyond basic syntax, many treebanks include enhanced dependencies that add arcs for semantic interpretation:

• Control relations: Making explicit the understood subject of infinitival clauses (e.g., 'She wants [her] to leave').
• Relative clauses: Adding a direct arc from the relativized noun to the predicate inside the relative clause.
• Coordination: Propagating shared dependents to all conjuncts. Some treebanks also include Abstract Meaning Representation (AMR) or semantic dependency layers, bridging syntax and compositional semantics.