Fuzzy Matching

The foundational edit distance metric that calculates the minimum number of single-character edits—insertions, deletions, or substitutions—required to transform one string into another.

• Mechanism: A dynamic programming matrix computes the cost of aligning two sequences character by character.
• Example: Transforming 'kitten' to 'sitting' requires 3 edits (substitute 'k' for 's', substitute 'e' for 'i', insert 'g').
• Use Case: Ideal for catching simple typos in names or addresses where keyboard proximity errors are common.
• Limitation: Treats all edit operations equally, failing to account for phonetic similarity or transposition frequency.

An extension of Levenshtein that adds adjacent character transposition as a single permitted operation, reflecting a common human typing error where two characters are swapped.

• Mechanism: The algorithm allows the operation cost[i][j] = cost[i-2][j-2] + 1 when characters at positions i and i-1 are swapped.
• Example: 'teh' and 'the' have a Levenshtein distance of 2 (delete 'e', insert 'e') but a Damerau-Levenshtein distance of 1 (transpose 'e' and 'h').
• Statistical Relevance: Studies show transpositions account for approximately 15-20% of all human typing errors, making this metric significantly more accurate for user-input validation.

A string comparator optimized for short strings like personal names, which assigns higher scores to strings that match from the beginning, based on the observation that name prefixes are rarely misspelled.

• Mechanism: The Jaro component counts matching characters within a sliding window, then applies a prefix scale p (typically 0.1) that boosts the score for strings sharing a common prefix of up to 4 characters.
• Formula: sim_w = sim_j + (l * p * (1 - sim_j)) where l is the length of the common prefix.
• Example: 'Martha' vs 'Marhta' scores 0.961, while 'Martha' vs 'Amrtha' scores significantly lower despite having the same edit distance.
• Industry Standard: The default matching algorithm in the U.S. Census Bureau's record linkage software for deduplicating person records.

A phonetic algorithm that indexes words by their sound rather than their spelling, encoding homophones to the same hash so that 'Smith' and 'Smyth' resolve to an identical code.

• Encoding Rules: Retains the first letter, then maps subsequent consonants to numeric codes (e.g., B/F/P/V → 1, D/T → 3), dropping vowels and silent letters.
• Example: Both 'Robert' and 'Rupert' encode to R163.
• Variants: Metaphone and Double Metaphone improve upon Soundex by incorporating English pronunciation rules and handling non-English origins, making them superior for diverse name matching.
• Application: Frequently used as a blocking key in record linkage to reduce the candidate comparison space before applying more expensive edit-distance calculations.

A vector-space approach that decomposes strings into overlapping character n-grams and measures the cosine of the angle between their term-frequency vectors, providing a token-order-aware similarity score.

• Mechanism: The string 'hello' broken into trigrams yields {'hel', 'ell', 'llo'}. The similarity is the dot product of the n-gram frequency vectors divided by the product of their magnitudes.
• Robustness: Highly resilient to word reordering and substring insertions that break edit-distance metrics.
• Example: 'John Smith' and 'Smith John' have a Levenshtein distance of 10 but a trigram cosine similarity of approximately 0.85.
• Integration: Often combined with TF-IDF weighting to down-weight common n-grams and amplify the signal from rare, discriminative character sequences.

The statistical gold standard for probabilistic record linkage that moves beyond simple similarity thresholds by calculating agreement and disagreement weights for each identity field.

• Mechanism: Estimates two conditional probabilities: m (the probability a field matches given the pair is a true match) and u (the probability a field matches by random chance). The match weight is log2(m/u).
• Decision Rule: Classifies pairs into matches, non-matches, and potential matches using an optimal threshold derived from error rate tolerances, with the potential matches flagged for clerical review.
• Example: A match on a rare surname like 'Zbikowski' receives a high positive weight, while a match on a common first name like 'John' contributes minimal evidence.
• Enterprise Context: Underpins modern identity resolution platforms by providing a mathematically rigorous framework for combining fuzzy name matching with deterministic signals like hashed emails.

The foundational string metric for fuzzy matching, calculating the minimum number of single-character edits—insertions, deletions, or substitutions—required to transform one string into another. It is the algorithmic backbone for catching typos in names or email addresses during record linkage.

• Example: 'John Smith' vs 'Jon Smith' has a Levenshtein distance of 1 (deletion of 'h').
• Application: Used to trigger merge suggestions when deterministic keys fail.

A rigid identity resolution method that requires exact, verified matches on personally identifiable information (PII), such as a hashed email or login credential. It provides absolute certainty but fails when data contains minor inconsistencies.

• Contrast: Fuzzy matching bridges the gap where deterministic logic breaks due to data entry errors.
• Use Case: Linking a logged-in session to a CRM record via a hashed email key.

A statistical approach that uses non-personal signals like IP address, browser type, and behavioral patterns to infer device ownership. It assigns a confidence score rather than a definitive link, often incorporating fuzzy matching to weigh the similarity of textual attributes.

• Synergy: Fuzzy matching algorithms contribute to the agreement weight for string fields in probabilistic models.
• Model: The Fellegi-Sunter framework formalizes this statistical linkage.

A centralized data structure that links all known identifiers—email addresses, device IDs, usernames—to a single unified customer profile. Fuzzy matching is the essential deduplication engine that prevents the creation of duplicate nodes for the same entity.

• Function: Serves as the backbone of cross-device personalization.
• Maintenance: Requires continuous fuzzy matching to merge new inbound data streams.

The definitive, best-version-of-the-truth customer profile created by applying survivorship rules to conflicting attributes from multiple source systems. Fuzzy matching identifies the disparate records that need to be merged, while survivorship logic selects the winning value.

• Process: 'John Smyth' (typo) is matched to 'John Smith' (CRM) via fuzzy logic, and the CRM value survives.
• Goal: A single, canonical view of the customer.

The process of algorithmically connecting multiple discrete web or app sessions into a single, continuous behavioral journey. Fuzzy matching helps link sessions where the user accidentally logged in with a slightly different username or email variant.

• Challenge: Timeouts and device switches fragment the user journey.
• Technique: Combines deterministic login data with fuzzy string comparison on session metadata.