Phrase Matching

Extends the standard inverted index by storing term positions within each document. Instead of just recording which documents contain a term, the postings list stores a list of integer offsets where the term occurs. This allows the engine to verify that query terms appear in the correct adjacent sequence rather than just anywhere in the document.

• Postings entry: term → [docID: [pos1, pos2, ...]]
• Enables O(1) adjacency checks during query execution
• Storage overhead is significant but essential for exact phrase queries

A relaxation of strict phrase matching controlled by a slop parameter that specifies the maximum allowed distance between terms. A slop of 0 requires exact adjacency and order. A slop of N allows terms to appear within N positions of each other, in any order.

• "machine learning"~2 matches "machine learning" and "learning machine"
• Balances precision and recall for natural language queries
• Implemented via edit distance on term position sequences

The retrieval engine processes a phrase query by intersecting the postings lists of each term and then scanning the positional data to find consecutive occurrences. The algorithm walks through candidate documents and checks if positions satisfy pos(term_i+1) = pos(term_i) + 1.

• Requires all terms to appear in the same document first
• Positional verification is a post-filtering step after Boolean intersection
• Can be optimized with skip pointers on positional data

An optimization technique that pre-indexes word n-grams as single tokens. Instead of storing individual term positions, the analyzer emits bigrams or trigrams as atomic units. A query for "neural network" becomes a single token lookup rather than a positional intersection.

• Bigram: "neural_network" indexed as one term
• Dramatically faster than positional verification at query time
• Trade-off: index size increases with n-gram vocabulary

Phrase matches can contribute to the overall relevance score beyond simple term frequency. Modern engines apply phrase boosts that multiply the score when terms appear in the exact query sequence. This rewards documents where the query appears as a coherent concept.

• BM25 score + phrase proximity bonus
• Common in multi-field scoring: higher weight for phrase match in title vs. body
• Prevents documents with scattered term occurrences from outranking exact matches

Phrase matching operates on lexical precision while semantic matching operates on conceptual similarity. A phrase query for "credit card fraud" will not match a document about "deceptive credit transactions" unless synonym expansion is applied. Hybrid systems combine both: phrase matching for exact entity names and semantic retrieval for paraphrased concepts.

• Phrase: high precision, zero recall for synonyms
• Semantic: high recall, risk of topical drift
• Combined via reciprocal rank fusion in hybrid search pipelines

The foundational data structure that makes phrase matching possible. An inverted index maps each unique term to a postings list containing document IDs and the exact term positions within each document.

• Stores positional data (e.g., word 5, word 12) for every term occurrence
• Enables the engine to verify that query terms appear adjacently
• Without positional indexing, only unordered bag-of-words matching is possible

The list of document identifiers and associated metadata for a specific term. For phrase matching, each posting includes positional information — the exact offset where the term appears.

• A query for "machine learning" retrieves postings for both terms
• The engine checks if 'learning' appears at position +1 relative to 'machine'
• Positional data significantly increases index size but is essential for adjacency verification

The method of segmenting text into tokens directly impacts phrase matching accuracy. Subword tokenization algorithms like Byte-Pair Encoding (BPE) can break phrases into unexpected units.

• Word-level tokenizers preserve natural phrase boundaries
• Subword tokenizers may split "unsupervised" into ["un", "super", "vised"]
• Mismatched tokenization between indexing and query time causes phrase match failures

The text processing pipeline that converts raw input into searchable tokens. Analyzers apply a tokenizer followed by a chain of filters (lowercasing, stemming, stop word removal).

• Stemming can break exact phrase matching by reducing terms to roots
• A phrase query may bypass stemming to preserve literal form
• Multi-field indexing stores both analyzed and raw text for flexible matching

A tokenization strategy that decomposes text into overlapping sequences of n characters or words. Bigram and trigram indexes enable robust phrase matching even with spelling errors.

• A bigram index for "phrase" produces ["ph", "hr", "ra", "as", "se"]
• Enables fuzzy phrase matching without exact term adjacency
• Increases recall at the cost of precision and index size

The retrieval paradigm that phrase matching belongs to — matching based on exact word overlap rather than semantic meaning. Contrasts with dense vector retrieval.

• Lexical matching guarantees precise term order when configured
• Fails on vocabulary mismatch where concepts use different words
• Modern hybrid systems combine lexical phrase matching with semantic retrieval for optimal precision and recall