Part-of-Speech Tagging (POS Tagging)

The earliest approach to POS tagging uses a hand-crafted lexicon and a set of disambiguation rules. A word is first looked up in a dictionary that lists all possible tags. Then, contextual rules—often expressed as regular expressions or constraint grammars—resolve ambiguities. For example, a rule might state: "If the preceding word is a determiner and the word can be a noun or verb, tag it as a noun." The Brill tagger is a classic example, starting with a lexicon and iteratively learning transformation rules. While interpretable, rule-based systems are brittle and labor-intensive to build for new languages or domains.

Stochastic taggers use Hidden Markov Models (HMMs) or Maximum Entropy Markov Models (MEMMs) to compute the most probable tag sequence. They model two probabilities:

• Transition probability: The likelihood of a tag given the previous tag(s), e.g., P(Verb|Noun).
• Emission probability: The likelihood of a word given a tag, e.g., P("run"|Verb).

The Viterbi algorithm efficiently finds the optimal tag path for a sentence. These models are trained on annotated corpora like the Penn Treebank and handle unseen words using suffix analysis or smoothing. They form the statistical backbone of many production taggers.

Modern POS taggers use deep learning architectures that eliminate manual feature engineering. A typical pipeline:

• Word Embeddings: Pre-trained vectors (Word2Vec, GloVe) capture semantic similarity.
• Contextual Encoder: A BiLSTM or Transformer reads the sentence bidirectionally to model context.
• CRF Output Layer: A Conditional Random Field on top ensures globally coherent tag sequences by modeling dependencies between adjacent output tags. Neural taggers excel at handling long-range dependencies and achieve state-of-the-art accuracy, especially when fine-tuned on domain-specific data.

To enable cross-lingual NLP, the Universal Dependencies (UD) project defines a standardized tag set of 17 universal POS tags. These include:

• NOUN, VERB, ADJ, ADV: Open-class content words.
• DET, ADP, CCONJ: Closed-class function words.
• PRON, PROPN: Pronouns and proper nouns.
• PUNCT, SYM, X: Punctuation, symbols, and other. This abstraction allows a single model architecture to be trained across dozens of languages, mapping language-specific fine-grained tags to the universal set.

POS tagging is fundamentally a disambiguation task. In English, many common words are highly ambiguous:

• "book" can be a NOUN ("read a book") or a VERB ("book a flight").
• "back" can be a NOUN, VERB, ADVERB, or ADJECTIVE. Taggers resolve this using contextual cues: the surrounding words, syntactic position, and morphological features. A word like "flies" is tagged as NOUN in "fruit flies" but VERB in "time flies". The accuracy on ambiguous words is the true measure of a tagger's quality, often significantly lower than overall token accuracy.

Beyond simple POS labels, advanced taggers assign detailed morphological features that are critical for lemmatization and parsing:

• Tense: Past, Present, Future.
• Number: Singular, Plural.
• Person: First, Second, Third.
• Case: Nominative, Accusative, Genitive.
• Gender: Masculine, Feminine, Neuter. For morphologically rich languages like Turkish or Finnish, a single word can encode many features. Taggers for these languages often use sequence-to-sequence models to predict the full morphological tag as a composite label, enabling accurate lemma recovery.

The task of identifying and classifying named mentions—such as persons, organizations, and locations—in unstructured text. NER systems heavily rely on POS tag sequences as features, as proper nouns typically exhibit distinct tagging patterns. A word tagged as NNP (proper noun, singular) is a strong signal for a potential entity boundary, making POS tagging an essential upstream component in entity extraction pipelines.

The computational parsing of a word into its constituent morphemes to identify its root form and grammatical features, including tense, number, person, and case. POS tagging and morphological analysis are deeply intertwined—many taggers jointly predict both the coarse POS category and fine-grained morphological features simultaneously, as the two tasks provide mutual constraints that improve overall accuracy.

The fundamental task of segmenting a continuous string of text into discrete tokens that serve as the input units for POS tagging. The tokenizer's decisions directly constrain the tagger's output—for example, splitting 'don't' into ['do', "n't"] versus keeping it as one token determines whether the tagger assigns separate verb and negation tags or a single contracted form. Consistent tokenization is therefore a prerequisite for reliable tagging.

A text normalization step that converts contracted forms like 'can't' to 'cannot' and 'we're' to 'we are'. This preprocessing step simplifies the POS tagger's task by eliminating ambiguous multi-token contractions that can confuse tag sequence models. Expanding contractions before tagging ensures that each syntactic word receives its own unambiguous POS label, improving downstream parsing accuracy.