Vector Similarity Search

The mathematical function used to quantify 'closeness' between two vectors fundamentally shapes retrieval results. The choice depends on the embedding model and the nature of the data.

• Cosine Similarity: Measures the angle between vectors, ignoring magnitude. Ideal for text embeddings where document length should not influence semantic similarity.
• Euclidean Distance (L2): Measures the straight-line distance. Sensitive to magnitude, making it suitable for scenarios where the absolute intensity of features matters, such as image color histograms.
• Dot Product: Combines angle and magnitude. Preferred for learned embeddings from models like BERT that are optimized for this metric.

Exact search over billions of high-dimensional vectors is computationally prohibitive. ANN algorithms trade a small amount of accuracy for massive speed gains by pre-indexing vectors into efficient data structures.

• HNSW (Hierarchical Navigable Small World): A graph-based algorithm that creates layered, skip-list-like structures for rapid traversal. It is the gold standard for high-recall, low-latency search.
• IVF (Inverted File Index): Partitions the vector space into clusters using k-means. A query only searches the nearest clusters, drastically reducing the search scope.
• LSH (Locality-Sensitive Hashing): Uses hash functions that place similar vectors into the same buckets with high probability, enabling sub-linear time retrieval.

High-dimensional vectors suffer from the 'curse of dimensionality,' where distance metrics lose discriminative power and computational cost explodes. Dimensionality reduction compresses vectors while preserving their relational structure.

• PCA (Principal Component Analysis): A linear technique that projects data onto the axes of maximum variance.
• Product Quantization (PQ): A lossy compression method that decomposes the high-dimensional space into smaller subspaces and quantizes each independently. This is a cornerstone of memory-efficient vector databases.
• Matryoshka Embeddings: A training technique that produces embeddings where a truncated prefix of the vector retains meaningful semantic power, allowing for flexible, on-the-fly dimensionality reduction without retraining.

Pure vector search can miss exact matches or fail to respect strict business logic. Hybrid search combines vector similarity with metadata filtering and lexical scoring to ensure both semantic breadth and keyword precision.

• Pre-filtering: Applies strict metadata constraints (e.g., date > 2024) before the vector search. This guarantees results meet criteria but can be slow if the filter is not selective.
• Post-filtering: Performs the vector search first and then removes results that don't match the filter. This is fast but risks returning fewer than the requested top_k results.
• Fusion (Reciprocal Rank Fusion): Combines a separate lexical (BM25) score and a vector similarity score into a single, re-ranked list, capturing both 'what is meant' and 'what is written'.

The effectiveness of a similarity search is entirely dependent on the embedding model that generated the vectors. The model must be aligned with the search task and data domain.

• Symmetric vs. Asymmetric: A symmetric model maps queries and documents to the same semantic space (e.g., all-MiniLM-L6-v2). An asymmetric model uses separate encoders for short queries and long documents, often yielding better results for Q&A over large texts.
• Domain Adaptation: A general-purpose model like text-embedding-3-large will underperform on specialized jargon (legal, medical) compared to a fine-tuned domain-specific model.
• Matryoshka Representation Learning: Models trained with this objective produce embeddings that are useful at multiple dimensions, allowing a single model to serve both high-accuracy and low-latency use cases.

Storing vectors as 32-bit floats consumes significant memory. Quantization reduces the bit-width of vector components, enabling massive scale with minimal accuracy loss.

• Scalar Quantization (SQ): Converts 32-bit floats to 8-bit or 4-bit integers, reducing memory by up to 4x.
• Binary Quantization (BQ): Compresses vectors to a single bit per dimension. A 1536-dimensional vector shrinks from ~6KB to ~192 bytes, enabling blazing-fast Hamming distance calculations in CPU cache.
• Geometric Quantization: A novel technique that maps vectors to a lattice of points on the unit sphere, preserving angular relationships better than scalar methods for cosine similarity searches.

A neural network that transforms unstructured data—text, images, or audio—into dense fixed-length vectors in high-dimensional space. These models are trained so that semantically similar inputs map to nearby points.

• Text embedders: BERT, Sentence-BERT, OpenAI text-embedding-3
• Multimodal embedders: CLIP, ImageBind
• Dimensionality: Typically 768, 1024, or 1536 dimensions

Example: The sentence 'cat sitting on a mat' and 'feline resting on a rug' produce vectors with high cosine similarity despite sharing no keywords.

Mathematical functions that quantify how close two vectors are in embedding space. The choice of metric fundamentally impacts retrieval quality and computational cost.

• Cosine Similarity: Measures the angle between vectors; range [-1, 1]. Most common for text embeddings. Ignores magnitude differences.
• Euclidean Distance (L2): Straight-line distance. Sensitive to vector magnitude. Preferred when absolute positioning matters.
• Dot Product: Equivalent to cosine similarity when vectors are normalized. Fastest to compute.
• Manhattan Distance (L1): Sum of absolute differences. Robust to outliers in sparse vectors.

A data structure that trades a small amount of recall for massive speed gains over brute-force search. Without ANN, querying billion-scale vector databases would be computationally prohibitive.

• HNSW (Hierarchical Navigable Small World): Multi-layer graph structure. Industry standard for high recall with low latency.
• IVF (Inverted File Index): Clusters vectors and searches only relevant clusters. Memory-efficient.
• PQ (Product Quantization): Compresses vectors for storage. Often combined with IVF as IVF-PQ.
• DiskANN: Designed for SSD-resident indexes. Enables search on vectors too large for RAM.

Typical tradeoff: 95-99% recall at 10-100x speedup vs. exact search.

The foundational retrieval operation: given a query vector, return the k most similar vectors from the index. This is the atomic unit of semantic search.

• Exact k-NN: Brute-force comparison against every vector. O(n*d) complexity. Guarantees perfect recall but scales poorly.
• Approximate k-NN: Uses an ANN index. Returns results in sub-millisecond time even on billion-scale datasets.
• Filtered k-NN: Applies metadata constraints (date ranges, categories) before or during vector search. Requires hybrid index strategies.

In RAG systems, k typically ranges from 3 to 20, balancing context window limits against retrieval completeness.

A purpose-built database system optimized for storing, indexing, and querying high-dimensional vector embeddings at scale. Distinct from traditional databases that index scalar values.

• Core capabilities: CRUD operations on vectors, ANN search, metadata filtering, and namespace isolation.
• Key systems: Pinecone, Weaviate, Milvus, Qdrant, pgvector (PostgreSQL extension).
• Index types supported: HNSW, IVF, DiskANN, and hybrid vector-scalar indexes.
• Typical scale: Production deployments handle 100M+ vectors with p99 latency under 100ms.

Vector databases are the memory backbone of RAG architectures, agentic systems, and semantic search engines.

A search paradigm that retrieves information based on conceptual meaning rather than keyword overlap. Powered by embedding models and vector similarity search.

• Keyword search: Matches 'car' to documents containing 'car'. Misses 'automobile'.
• Semantic search: Matches 'car' to documents about 'automobiles', 'vehicles', and 'sedans'.
• Hybrid search: Combines vector similarity with BM25 keyword scoring. Addresses the 'keyword-not-in-vocabulary' failure mode of pure semantic search.
• Use cases: Enterprise document retrieval, product recommendation, customer support ticket routing, and legal e-discovery.

Modern RAG pipelines rely on semantic search to retrieve relevant context chunks before generation.