Cosine Similarity

Cosine similarity calculates the cosine of the angle between two vectors, not the Euclidean distance. This makes it magnitude-invariant: a document vector scaled to double its length will have a similarity of 1.0 with its original. This property is critical for comparing text embeddings where document length should not dictate semantic relevance.

• Range: [-1, 1] (for vectors with negative components)
• Range: [0, 1] for standard non-negative embeddings
• Value of 1: Identical orientation
• Value of 0: Orthogonal (no similarity)
• Value of -1: Diametrically opposed

The formula is cos(θ) = (A · B) / (||A|| * ||B||). The dot product in the numerator captures the co-occurrence of features. The L2 norms in the denominator normalize the result. In high-dimensional spaces, this is equivalent to the Pearson correlation coefficient when vectors are centered. For unit vectors (normalized to length 1), cosine similarity reduces to a simple dot product, enabling massive computational speedups in vector databases.

In very high-dimensional spaces (e.g., 768+ dimensions from BERT embeddings), random vectors tend to be nearly orthogonal. Cosine similarity helps mitigate this by focusing on angular distance, but it is not immune. All points in a high-dimensional hypersphere concentrate near the surface. This is why approximate nearest neighbor (ANN) algorithms like HNSW are essential—exact brute-force cosine similarity search becomes computationally prohibitive and less discriminative without indexing.

Cosine similarity is the default metric for comparing dense vector embeddings from models like OpenAI's text-embedding-3, Sentence-BERT, and CLIP. When a user searches a vector database, the query is embedded into the same space, and cosine similarity scores rank the results.

• Text: Semantic textual similarity
• Images: CLIP embeddings for text-to-image search
• Audio: Acoustic fingerprint matching
• Code: CodeBERT for clone detection

Choosing the right similarity metric depends on the embedding model's training objective. Cosine similarity is preferred when the model was trained with a contrastive loss or cosine-based softmax. Euclidean distance is appropriate when magnitude matters. Dot product is used when vectors are already normalized.

• If embeddings are L2-normalized, Euclidean distance and cosine similarity have a monotonic relationship: ||A-B||² = 2(1 - cos(θ))
• Inner product can be skewed by vector magnitude, making it less reliable for raw embeddings.

In content deduplication pipelines, perceptual hashes like NeuralHash or SimHash produce compact binary vectors. Cosine similarity (or its binary equivalent, Hamming distance) compares these fingerprints. A threshold of > 0.95 typically flags near-duplicate content. This technique underpins YouTube's Content ID and Apple's CSAM detection, where a query fingerprint is compared against a database of known hashes using efficient ANN indexing.