Part-of-Speech Tagging

The tagset defines the inventory of grammatical labels a system can assign. Tagsets range from coarse-grained to fine-grained:

• Universal POS Tags (UPOS): 17 cross-linguistic categories (NOUN, VERB, ADJ, ADP) designed for multilingual consistency
• Penn Treebank Tags: 36+ fine-grained tags distinguishing singular vs. plural nouns (NN vs. NNS) and verb forms (VB, VBD, VBG, VBN, VBP, VBZ)
• Language-Specific Extensions: Morphologically rich languages often require hundreds of tags encoding case, gender, and number

The choice of tagset directly impacts the information available to a dependency parser for arc labeling.

Lexical ambiguity is the central challenge in POS tagging. Many word forms can serve multiple grammatical roles:

• Homograph resolution: The word 'run' can be a verb ('they run daily') or a noun ('a production run')
• Contextual disambiguation: A Hidden Markov Model or CRF uses surrounding tokens to resolve ambiguity
• Closed-class reliability: Determiners ('the', 'a') and prepositions ('of', 'in') are highly reliable anchors because they belong to closed, unambiguous classes

Modern neural taggers use contextualized embeddings from transformers to capture the syntactic role of ambiguous words with over 97% accuracy on standard benchmarks.

Beyond simple category assignment, advanced POS tagging systems encode rich morphological features that are essential for syntactic agreement:

• Tense and Aspect: Distinguishing past ('walked'), present ('walks'), and future ('will walk') verb forms
• Person and Number: Encoding first-person singular ('I am') vs. third-person plural ('they are')
• Case Marking: Nominative ('he') vs. accusative ('him') for pronoun resolution
• Gender and Animacy: Critical for languages like French, German, and Russian where adjectives and determiners agree with noun gender

The Universal Dependencies framework standardizes these features using the UniMorph schema, enabling consistent annotation across treebanks.

POS tagging is fundamentally a sequence labeling task where each token's prediction depends on its neighbors. Three generations of approaches exist:

• HMM and CRF Models: Statistical models that learn transition probabilities between adjacent tags (e.g., a determiner is likely followed by an adjective or noun)
• BiLSTM-CRF Architectures: Neural networks that encode bidirectional context and use a Conditional Random Field output layer to enforce valid tag sequences
• Transformer Fine-Tuning: Models like BERT and XLM-RoBERTa produce contextualized token representations that implicitly capture syntactic function, achieving state-of-the-art accuracy without explicit CRF decoding

The transition from discrete feature engineering to learned representations eliminated the need for hand-crafted suffix features and capitalization patterns.

POS tagging accuracy is tightly coupled to the preceding tokenization step. Errors at token boundaries propagate directly:

• Multi-word expressions: 'New York' should be treated as a single PROPN token, not separately tagged as ADJ + PROPN
• Clitic splitting: In Romance languages, 'darlo' ('to give it') must be segmented into verb 'dar' + clitic 'lo' before tagging
• Subword tokenization conflicts: BPE tokenization from transformer models can split a single linguistic word into multiple subword tokens, requiring alignment strategies to map subword predictions back to word-level tags
• Contractions: English 'don't' requires decomposition into 'do' (AUX) and 'n't' (PART) for accurate dependency parsing

Production systems must ensure the tokenizer and tagger share consistent word boundary definitions.

POS tags serve as the primary input features for dependency parsers. Tagging errors cascade into structural errors:

• Head selection: A verb incorrectly tagged as a noun will not be identified as the syntactic root of a clause
• Arc labeling: Distinguishing nsubj (nominal subject) from dobj (direct object) depends on correctly identifying the verb and its argument nouns
• Prepositional phrase attachment: Correctly tagging the preposition (ADP) and its object noun constrains the parser's attachment decision
• Error propagation measurement: Studies show that a 1% degradation in POS accuracy can cause a 2-3% drop in Labeled Attachment Score (LAS)

This tight coupling motivates joint models that perform tagging and parsing simultaneously rather than in a pipeline.