MinHash is a locality-sensitive hashing (LSH) technique that estimates the Jaccard similarity between two sets by comparing only a small, fixed-size sketch of minimum hash values. It applies multiple hash functions to each element of a set and retains only the minimum hash value per function, creating a compact signature that preserves similarity while dramatically reducing computational complexity.
Glossary
MinHash
A locality-sensitive hashing technique that estimates the Jaccard similarity between two sets of k-mers by comparing only a small, fixed-size sketch of minimum hash values, enabling rapid, large-scale genomic distance estimation.
LOCALITY-SENSITIVE HASHING
What is MinHash?
A dimensionality reduction technique for estimating set similarity without comparing every element.
In genomics, MinHash operates on sets of k-mers to rapidly estimate genomic distance without performing expensive base-by-base alignment. Tools like Mash and sourmash use this approach to compare thousands of genomes in seconds, enabling large-scale tasks such as taxonomic classification, metagenomic sample clustering, and identifying nearest neighbors in reference databases.
LOCALITY-SENSITIVE HASHING
Key Features of MinHash
MinHash is a probabilistic technique that compresses large sets of k-mers into compact, fixed-size sketches, enabling ultra-fast estimation of Jaccard similarity without pairwise sequence alignment.
01
Jaccard Similarity Estimation
MinHash estimates the Jaccard index between two sets by comparing their sketches. The probability that two sketches share the same minimum hash value equals the Jaccard coefficient of the original sets.
- Formula: J(A,B) = |A ∩ B| / |A ∪ B|
- Unbiased estimator: The fraction of matching hashes across k independent permutations
- Error bound: O(1/√k), where k is the sketch size
- Example: A sketch size of 1,000 yields a standard error of approximately 3%
02
k-mer Set Compression
MinHash reduces a genome represented by millions of overlapping k-mers (typically k=21 or k=31) into a compact sketch of only a few hundred or thousand integer values.
- Input: A set S containing all canonical k-mers from a sequencing read or genome
- Output: A fixed-size array of minimum hash values, independent of genome size
- Compression ratio: A 5 Mbp bacterial genome with ~5 million k-mers can be sketched to 1,000 integers
- Storage: Sketches require only kilobytes, enabling in-memory comparison of millions of genomes
03
Locality-Sensitive Property
MinHash belongs to the family of Locality-Sensitive Hashing (LSH) algorithms, meaning similar input sets map to similar sketches with high probability.
- Collision probability: Directly proportional to set similarity
- Hash function requirement: Must be a universal or pairwise-independent hash function
- Common implementations: MurmurHash3, xxHash, or SHA-1 truncated to 64-bit integers
- Canonical k-mers: Both forward and reverse-complement k-mers are hashed, and the minimum is retained to ensure strand invariance
04
Mash Distance Calculation
The Mash tool extends MinHash to compute a robust evolutionary distance metric between genomes, approximating the Average Nucleotide Identity (ANI).
- Mash distance formula: D = -(1/k) * ln(2j / (1+j)), where j is the estimated Jaccard index and k is the k-mer size
- ANI relationship: ANI ≈ 1 - D under simple mutation models
- Thresholds: Mash distance < 0.05 corresponds to ~95% ANI, the standard species boundary
- Performance: Comparing two bacterial genomes takes milliseconds, versus minutes for whole-genome alignment
05
FracMinHash for Abundance Filtering
FracMinHash extends standard MinHash by retaining only hash values that fall below a specified abundance threshold, enabling robust containment estimation in metagenomic samples.
- Mechanism: A k-mer's hash is included in the sketch only if its multiplicity in the dataset exceeds a minimum count
- Containment index: Estimates what fraction of k-mers from a query genome are present in a mixture
- Application: Identifying low-abundance pathogens in complex metagenomes without assembling genomes
- Tools: Implemented in sourmash and scaled MinHash signatures for large-scale microbiome comparisons
06
Streaming and Distributed Computation
MinHash sketches are composable and mergeable, enabling distributed computation across clusters and incremental updates without reprocessing raw data.
- Merge operation: The sketch of a combined dataset is the element-wise minimum of individual sketches
- Streaming: Sketches can be updated one k-mer at a time, requiring only O(sketch_size) memory
- Parallelization: Individual sequencing runs or genome partitions can be sketched independently and merged later
- Use case: Real-time taxonomic classification of nanopore sequencing streams by comparing streaming sketches against reference databases
LOCALITY-SENSITIVE HASHING COMPARISON
MinHash vs. Alternative Sketching Methods
Comparative analysis of MinHash against alternative sketching and dimensionality reduction techniques used for large-scale genomic distance estimation and set similarity computation.
| Feature | MinHash | FracMinHash | HyperLogLog | SimHash |
|---|---|---|---|---|
Core Objective | Jaccard similarity estimation | Containment index estimation | Cardinality estimation | Cosine similarity estimation |
Hash Selection Strategy | Minimum k hash values | Hash values below abundance threshold | Maximum leading zeros in hash binary representation | Random hyperplane projections |
Output Data Structure | Fixed-size sketch of minimum hashes | Fractional sketch of low-value hashes | Single probabilistic counter | Fixed-width binary fingerprint |
Supports Abundance Weighting | ||||
Sensitive to Set Size Disparity | ||||
Memory Complexity | O(sketch size) | O(sketch size × fraction) | O(log log n) | O(fingerprint width) |
Typical Sketch Size | 1,000-10,000 hashes | Variable; 1-10% of k-mers | ~1.5 KB per counter | 64-256 bits |
Query Throughput |
|
|
|
|
MINHASH EXPLAINED
Frequently Asked Questions
Clear, technical answers to the most common questions about MinHash, its mechanisms, and its role in large-scale genomic similarity estimation.
MinHash is a locality-sensitive hashing (LSH) technique that estimates the Jaccard similarity between two sets by comparing only a small, fixed-size sketch of minimum hash values. The core mechanism works by applying multiple independent hash functions to each element in a set (e.g., k-mers from a genome) and recording only the minimum hash value produced by each function. This collection of minima forms the MinHash sketch. The probability that two sets produce the same minimum hash value for a given hash function is exactly equal to their Jaccard index. By using, for example, 1,000 hash functions, the fraction of matching minima between two sketches provides an unbiased estimator of the true set overlap. This reduces the computational complexity of comparing two sets from O(n+m) to O(k), where k is the sketch size, enabling rapid, large-scale genomic distance estimation without storing or comparing the original sets.
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.
Enterprise searchRAGPermissions
Read more
Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.
AI agentsWorkflow automationGovernance
Read more
Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
AI integrationDecision supportModel routing
Read more
About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
LinkedIn
Limited slotsGet a Free AI Consultation
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us