Bloom Filter

A Cuckoo filter is a probabilistic data structure that, like a Bloom filter, tests for set membership but offers additional functionality. Its key advantages include:

• Support for deletions of items, which standard Bloom filters cannot do efficiently.
• Lower space overhead for the same false positive rate in many practical scenarios.
• It stores a compact fingerprint of each item in one of a small number of possible buckets using cuckoo hashing.

While slightly more complex, it's often preferred when dynamic sets require item removal.

A Count-Min Sketch is a probabilistic data structure used to estimate the frequency of events in a data stream. It is a core tool for streaming algorithms where storing exact counts is infeasible.

• It uses multiple hash functions to map items to an array of counters, incrementing each.
• To query an item's frequency, it takes the minimum value across all its associated counters, which provides an upper-bound estimate.
• It is characterized by its controllable error rate and confidence parameter.

This makes it ideal for tracking heavy hitters or approximating counts in high-throughput systems.

HyperLogLog is a probabilistic algorithm for the count-distinct problem, estimating the cardinality (number of unique elements) of extremely large multisets with minimal memory.

• It works by observing the maximum number of leading zeros in the binary representation of hashed items.
• The core insight is that the probability of seeing a hash with many leading zeros is low, and this probability can be used to estimate the number of unique items.
• It provides estimates with a typical error rate of about 1-2% while using only kilobytes of memory, even for billions of items.

It is a cornerstone of big data analytics in databases like Redis and Presto.

A Quotient filter is a compact, probabilistic hash table that supports approximate membership queries, insertions, and deletions. It is often compared to Bloom and Cuckoo filters.

• It stores a fingerprint for each item, partitioned into a quotient (used as an index) and a remainder.
• It uses a variant of linear probing to handle collisions, enabling high space utilization.
• It supports merging of filters and provides better locality of reference than standard Bloom filters, potentially offering faster lookups.

It is particularly useful in database and storage systems for metadata indexing.

Lossy Counting is a deterministic, one-pass streaming algorithm for finding frequent items (items exceeding a support threshold) in a data stream. It trades some accuracy for bounded memory use.

• It processes the stream in windows and maintains counts for observed items, decrementing all counts at the end of each window.
• This process guarantees that all true frequent items are found, but some infrequent items (false positives) may also be reported.
• The error bound is a function of the window size and is known in advance.

It is a foundational algorithm for real-time analytics where Bloom filters or sketches might be used for related frequency estimation tasks.

MinHash (or the min-wise independent permutations locality-sensitive hashing scheme) is a technique for quickly estimating the similarity (Jaccard index) between two sets. It is a form of dimensionality reduction for sets.

• It works by generating several hash functions, and for each, storing the minimum hash value of all items in a set. This collection of minima is the "signature."
• The similarity between two sets is approximated by the fraction of matching minima in their signatures.
• Like Bloom filters, it provides a compact, fixed-size representation of a set, but optimized for similarity search rather than membership queries.

It is widely used for near-duplicate detection, web clustering, and large-scale recommendation systems.