Prepositional Phrase Attachment

PP attachment ambiguity arises when a prepositional phrase can logically modify either the preceding verb or the preceding noun phrase. In the classic example 'I saw the man with the telescope', the PP 'with the telescope' can attach to the verb 'saw' (instrumental reading: using a telescope to see) or to the noun 'the man' (attributive reading: the man who has a telescope). This structural ambiguity creates two distinct parse trees with different semantic interpretations.

The canonical formulation for statistical PP attachment disambiguation uses a four-head lexicalized model consisting of:

• v: The head verb of the verb phrase
• n1: The head noun of the object noun phrase
• p: The preposition itself
• n2: The head noun inside the prepositional phrase

For 'eat pizza with fork', the model evaluates whether with attaches to eat (v) or pizza (n1), using fork (n2) as additional evidence.

Simple heuristics provide surprisingly strong baselines for this task:

• Right Association: Attach the PP to the most recent noun phrase (n1 attachment)
• Verb Default: Always attach to the verb (v attachment)
• Frequency Baseline: Choose the attachment seen most often for a given (v, n1, p) triple in training data

These heuristics achieve approximately 70-75% accuracy, while state-of-the-art neural models reach 85-90% by incorporating richer contextual representations.

Disambiguation often requires real-world knowledge beyond syntax:

• 'eat pizza with fork' → Verb attachment (forks are instruments for eating)
• 'eat pizza with pepperoni' → Noun attachment (pepperoni is a pizza topping)
• 'eat pizza with friends' → Verb attachment (friends are companions for eating)

The semantic type of n2 relative to v and n1 drives the decision. Knowledge graphs and distributional semantics help models learn these selectional preferences.

Incorrect PP attachment propagates errors into critical NLP applications:

• Relation Extraction: 'CEO of Apple announced...' must correctly attach of Apple to CEO to extract the correct affiliation
• Question Answering: 'Who wrote the book on the table?' requires resolving whether on the table modifies book or wrote
• Machine Translation: Different attachment decisions produce different syntactic structures in the target language

Modern systems use contextualized embeddings from models like BERT to capture the interaction between all four lexical heads simultaneously. Key architectural patterns include:

• Biaffine attention between the PP head and candidate attachment sites
• Graph neural networks that jointly model all PP attachments in a sentence
• Multi-task learning combining PP attachment with related tasks like semantic role labeling

These approaches leverage pre-trained language models' implicit world knowledge learned from massive corpora.

The foundational syntactic analysis task that constructs a directed graph of grammatical relationships between words. PP attachment is a primary source of ambiguity in these structures.

• Head-Dependent Relations: Identifies which word governs another.
• Typed Dependencies: Labels relations like nmod (nominal modifier) or obl (oblique).
• Ambiguity Resolution: Must decide if "saw the man with the telescope" attaches the PP to the verb or the noun.

A parsing paradigm that scores all possible dependency arcs globally to find the maximum spanning tree. This global view helps resolve PP attachment by considering the entire sentence context.

• Maximum Spanning Tree: Uses the Chu-Liu/Edmonds algorithm for decoding.
• Global Features: Can incorporate long-distance information to disambiguate modifiers.
• Arc-Factored Models: Assume independence between arcs for tractability, but higher-order models capture richer contexts.

Moves beyond surface syntax to capture who did what to whom. This task often abstracts away from syntactic alternations to resolve the underlying meaning of PP attachments.

• Predicate-Argument Structures: Identifies semantic roles rather than grammatical functions.
• Abstract Meaning Representation (AMR): Encodes meaning as a rooted, directed graph where PPs are attached to the correct conceptual node.
• Enhanced Dependencies: Augment basic syntax with implicit relations to clarify attachment.

A deterministic parsing paradigm that processes sentences left-to-right using a stack and buffer. PP attachment decisions are made incrementally via shift-reduce actions.

• Arc-Eager Strategy: Builds dependencies as soon as the dependent is complete, making early attachment commitments.
• Dynamic Oracle: Trains parsers to recover from erroneous attachment decisions during exploration.
• Beam Search: Maintains multiple partial parse states to mitigate error propagation from greedy PP attachment choices.

The primary evaluation metric for dependency parsers. It measures the percentage of tokens assigned both the correct syntactic head and the correct dependency label.

• PP Attachment Precision: Directly impacts LAS, as incorrect head assignment for a prepositional phrase penalizes the score.
• Unlabeled Attachment Score (UAS): Measures only correct head assignment, ignoring the relation label.
• Benchmarking: State-of-the-art neural parsers achieve high LAS on standard treebanks by learning semantic cues for attachment.

A neural architecture by Dozat and Manning that uses deep biaffine attention over BiLSTM-encoded word representations. It excels at PP attachment by learning rich lexical and contextual features.

• Biaffine Attention: Applies a bilinear transformation to score all possible head-dependent pairs globally.
• Contextualized Embeddings: Leverages deep representations to capture the semantic affinity between a preposition and its correct head.
• State-of-the-Art: Achieved leading graph-based parsing accuracy by effectively modeling attachment preferences.