Winograd Schema

A Winograd Schema consists of a pair of sentences that are identical except for one or two words that completely flip the antecedent of a pronoun. This minimal edit forces the system to rely on world knowledge rather than lexical co-occurrence. For example:

• The city councilmen refused the demonstrators a permit because they feared violence. (they = councilmen)
• The city councilmen refused the demonstrators a permit because they advocated violence. (they = demonstrators) The change from 'feared' to 'advocated' inverts the pronoun's referent.

Resolution cannot be achieved through selectional restrictions or simple syntactic parsing. The correct antecedent is determined by deep, implicit knowledge about the world:

• Understanding that councilmen are typically responsible for public safety and thus more likely to fear violence.
• Knowing that demonstrators are more likely to advocate for change, potentially including disruptive actions. This reliance on script-based knowledge and naive physics makes the schemas robust against shallow statistical solutions.

Proposed by Hector Levesque in 2011 as an alternative to the Turing Test, the WSC was a dataset of 273 schemas designed to be Google-proof. The key design principles were:

• Questions must be easily answerable by a human adult.
• Questions must not be solvable by simple statistical text analysis or web search.
• The correct answer must require reasoning about events, causality, and social norms. The challenge was officially retired in 2019 after transformer-based models exceeded human baseline performance, though debate continues about whether true reasoning was achieved.

Winograd Schemas are a highly constrained subset of pronominal anaphora. Unlike general coreference resolution, they specifically target:

• Ambiguous pronouns with two grammatically plausible antecedents.
• Minimal pairs that isolate a single semantic variable.
• Deliberate neutralization of gender and number agreement cues. This specialization makes them a pure test of pragmatic reasoning rather than syntactic feature engineering, distinguishing them from the broader task of coreference chain construction.

The schemas are explicitly constructed to foil bag-of-words models and co-occurrence statistics. In the original example:

• The trophy doesn't fit in the suitcase because it is too big. (it = trophy)
• The trophy doesn't fit in the suitcase because it is too small. (it = suitcase) A statistical system cannot rely on the word 'fit' or 'trophy' to resolve 'it', because the context words are identical. The system must understand the physical dynamics of containment—that a container must be larger than the object it holds.

Winograd Schemas represent a diagnostic subset within the broader field of coreference resolution. While standard coreference systems handle entity linking across a document, Winograd Schemas isolate the specific failure mode of commonsense integration. A model that achieves high F1 scores on CoNLL-2012 may still fail on Winograd Schemas if it relies on mention-pair features rather than building a discourse-level situation model. This highlights the gap between surface-level coreference and deep semantic understanding.

A linguistic expression whose interpretation depends on a preceding expression. In the Winograd Schema, the pronoun is an anaphor that must be resolved to the correct antecedent.

• Example: 'The city council refused the demonstrators a permit because they feared violence.'
• The pronoun 'they' is anaphoric, requiring world knowledge to determine if it refers to the council or the demonstrators.
• Differs from cataphora, where the pronoun precedes its referent.

The specific subtask of coreference resolution focused exclusively on resolving pronouns to their correct antecedent noun phrases. Winograd Schemas are a direct test of this capability.

• Requires deep semantic understanding rather than simple gender or number agreement.
• Traditional systems relied on salience models and Centering Theory.
• Modern approaches use mention-ranking models with SpanBERT representations.

The capacity to use implicit, real-world knowledge to make inferences. Winograd Schemas are designed specifically to evaluate this ability in AI systems.

• Example: 'The trophy doesn't fit in the suitcase because it's too big.'
• Humans instantly know 'it' refers to the trophy, not the suitcase, based on typical object sizes.
• This requires physical, social, and procedural world knowledge that cannot be derived from syntax alone.

A neural coreference architecture that scores all candidate antecedents for a given mention and selects the highest-ranked one. This is the dominant approach for tackling Winograd-style challenges.

• Uses biaffine attention to compute pairwise scores between the pronoun and each candidate span.
• Higher-order inference iteratively refines span representations based on predicted antecedents.
• The canonical e2e-coref model by Lee et al. jointly performs mention detection and coreference scoring.

A pre-training method for BERT that masks contiguous spans of tokens and predicts them using span boundary representations. It is optimized for span-level tasks like coreference resolution.

• Outperforms standard BERT on Winograd Schema benchmarks by learning better span representations.
• The model learns to predict the entire masked span from the vectors of its boundary tokens.
• Provides the foundational embeddings used by state-of-the-art neural coreference systems.

A syntactic theory governing the distribution of anaphors, pronominals, and referring expressions. It provides structural constraints that can eliminate impossible antecedents in Winograd Schemas.

• Principle A: Anaphors (reflexives) must be bound within their local domain.
• Principle B: Pronominals must be free within their local domain.
• Principle C: Referring expressions must be free everywhere.
• These constraints serve as features in deterministic coreference and rule-based sieve architectures.