Cosine Similarity

Cosine similarity normalizes for vector length, making it magnitude-agnostic. Two documents with identical semantic content but different lengths (e.g., a paragraph vs. a full page) will produce embeddings with different magnitudes but similar directions. Cosine similarity correctly identifies them as related by measuring only the angle between them.

• Formula: cos(θ) = (A · B) / (||A|| × ||B||)
• Range: -1 (opposite) to 1 (identical), with 0 indicating orthogonality
• Key insight: A document repeated 10 times yields the same cosine match as a single occurrence

While dot product captures both magnitude and direction, cosine similarity strips magnitude away. In dense retrieval, cosine is preferred because embedding magnitudes can vary due to document length or model quirks, introducing noise into similarity rankings.

• Dot product: A · B = ||A|| × ||B|| × cos(θ)
• When to use dot product: When magnitude carries meaningful signal (e.g., term importance in some models)
• When to use cosine: When you want pure semantic alignment, which is the default for most modern embedding models like text-embedding-3 or all-MiniLM-L6-v2

When all vectors are L2-normalized to unit length (magnitude = 1), cosine similarity becomes numerically equivalent to the dot product. This is a critical optimization: vector indexes like FAISS and HNSW implementations often store pre-normalized vectors to reduce cosine similarity to a simple dot product at query time.

• Normalization: v̂ = v / ||v||
• Result: cos(θ) = v̂₁ · v̂₂ (pure dot product after normalization)
• Practical impact: Eliminates division operations during search, reducing latency in high-throughput retrieval pipelines

Cosine similarity can be directly converted to angular distance, which is a proper distance metric satisfying the triangle inequality. This is useful when you need a true distance function for clustering algorithms or when interpreting similarity in human-intuitive terms.

• Angular distance: d(A, B) = arccos(cosine_similarity) / π
• Range: 0 (identical) to 1 (opposite)
• Use case: K-means clustering on embeddings often uses angular distance because it respects the geometry of the normalized hypersphere

In high-dimensional spaces (typical embedding dimensions range from 384 to 3072), cosine similarity exhibits counterintuitive properties. Random vectors tend toward orthogonality (cosine ≈ 0), making the similarity score distribution highly concentrated. This is why approximate nearest neighbor (ANN) indexes are essential—exhaustive search becomes both computationally prohibitive and semantically noisy.

• Curse of dimensionality: As dimensions increase, the contrast between nearest and farthest neighbors diminishes
• Mitigation: ANN algorithms like HNSW exploit this structure to prune search space efficiently
• Practical threshold: Cosine > 0.7 typically indicates strong semantic relatedness in well-trained embedding spaces

While cosine similarity ranges from -1 to 1, many vector databases and ANN libraries clamp or ignore negative values because they assume non-negative embedding spaces (common with ReLU-activated models). Understanding your model's output distribution is critical for index configuration.

• ReLU embeddings: All values ≥ 0, so cosine is always in [0, 1]
• GELU/SiLU embeddings: Can produce negative values, requiring full [-1, 1] range support
• Index configuration: Some ANN indexes like IVF-PQ assume inner product space; using cosine requires explicit normalization or metric selection (e.g., MetricType.COSINE in Milvus)

While Euclidean distance measures the straight-line magnitude between two vector endpoints, cosine similarity exclusively measures angular difference. In high-dimensional sparse spaces, magnitude often correlates with document length or word frequency rather than semantic relevance. Cosine similarity normalizes for this by focusing solely on directional alignment, making it the default choice for text embeddings where document length should not influence relevance scoring. However, for normalized embeddings (unit vectors), Euclidean distance and cosine similarity produce equivalent rankings due to the mathematical relationship: cosine_sim = 1 - (euclidean_dist² / 2).

The dot product is the unnormalized precursor to cosine similarity. It multiplies corresponding vector components and sums the results, but does not divide by vector magnitudes. This means longer documents or more frequent terms naturally produce higher dot product scores, which can be advantageous when magnitude carries meaningful signal—such as prioritizing comprehensive documents. Many embedding models are trained with dot product objectives, and some vector databases default to dot product scoring for raw performance. Cosine similarity is simply the dot product of L2-normalized vectors.

Modern vector databases like Milvus, Qdrant, and Weaviate implement cosine similarity as inner product search on pre-normalized vectors. The normalization step projects all vectors onto the unit hypersphere, after which inner product becomes mathematically equivalent to cosine similarity. This optimization is critical because:

• ANN algorithms like HNSW are natively optimized for inner product or L2 distance
• Pre-normalization avoids costly division operations during query time
• Index build times increase slightly, but query latency drops significantly
• Some databases support hybrid scoring that blends cosine similarity with metadata boosts

Cosine similarity ranges from -1 to +1, where -1 indicates diametrically opposed vectors. In practice, most text embedding models trained with contrastive objectives produce vectors in the positive orthant, rarely yielding negative similarities. However, negative values can emerge when:

• Embedding models are trained with explicit negative pairs
• Queries and documents belong to genuinely antithetical domains
• Using additive angular margin penalties during training Negative cosine similarity can serve as a hard exclusion filter, automatically rejecting documents that are semantically opposite to the query intent.

Contrastive learning frameworks like SimCSE, CLIP, and DPR use cosine similarity as the core scoring function in their InfoNCE loss. The loss maximizes cosine similarity between positive pairs (query and relevant document) while minimizing it for negative pairs (query and irrelevant documents) using a temperature-scaled softmax. Key design decisions include:

• Temperature parameter: Controls the concentration of the similarity distribution
• In-batch negatives: Other items in the mini-batch serve as negative examples
• Hard negative mining: Selecting negatives with moderate cosine similarity to improve discriminative power The choice of cosine similarity over dot product ensures training stability across varying embedding magnitudes.

Angular distance is the direct geometric interpretation of cosine similarity, computed as arccos(cosine_similarity) / π, producing a value between 0 (identical direction) and 1 (opposite direction). This metric is a proper distance function satisfying the triangle inequality, unlike cosine similarity itself. Angular distance is preferred when:

• Clustering embeddings using algorithms that require metric spaces
• Computing geodesic distances on the unit hypersphere
• Building visualization projections like t-SNE or UMAP
• Performing spherical k-means clustering on normalized document embeddings