Cosine Similarity Threshold

Cosine similarity measures the cosine of the angle between two embedding vectors in high-dimensional space, producing a value between -1 and 1. A score of 1.0 indicates identical orientation (perfect semantic match), 0 indicates orthogonality (no relationship), and -1 indicates opposite meaning. Unlike Euclidean distance, cosine similarity is magnitude-invariant, making it robust to variations in embedding vector length that arise from different context window sizes or token counts. This property is essential for comparing a single abbreviation mention against multiple candidate sense embeddings of varying norms.

The threshold value directly governs the balance between false positives and false negatives in abbreviation disambiguation. A high threshold (e.g., 0.90) enforces strict semantic matching, maximizing precision but risking missed correct mappings when contextual embeddings are noisy. A low threshold (e.g., 0.60) increases recall by accepting more candidate senses but introduces incorrect expansions. Clinical NLP systems often employ task-specific thresholds—a higher bar for safety-critical concepts like medication names, and a lower bar for general clinical findings where downstream human review exists.

Raw cosine similarity scores are not inherently calibrated probabilities. A score of 0.80 does not mean an 80% chance of correctness. To address this, clinical disambiguation pipelines often apply softmax normalization across all candidate senses, converting similarity scores into a probability distribution where all values sum to 1.0. This enables confidence thresholding—rejecting all candidates when the top probability falls below a minimum (e.g., 0.50), flagging the mention for human review. Platt scaling or isotonic regression can further calibrate these probabilities using a held-out validation set.

A single global threshold is often suboptimal because different UMLS semantic types exhibit varying degrees of embedding space density. For example, 'Clinical Drug' senses (e.g., 'ASA' for Aspirin vs. 5-Aminosalicylic Acid) are tightly clustered and require a higher threshold to separate. In contrast, 'Disease or Syndrome' senses may be more dispersed, allowing a lower threshold. Advanced systems implement type-aware thresholding, where the minimum cosine similarity is dynamically selected based on the semantic type of the candidate senses retrieved from the UMLS Metathesaurus.

The optimal cosine similarity threshold is not portable across embedding models. A threshold of 0.85 tuned for ClinicalBERT embeddings will not directly transfer to PubMedBERT or a general-domain model like text-embedding-3-large. Each model produces embeddings in a different latent space geometry, with varying degrees of anisotropy—the tendency for all embeddings to concentrate in a narrow cone. Models with high anisotropy produce cosine similarities clustered near 1.0, requiring higher thresholds. Threshold calibration must be repeated whenever the underlying embedding model is changed or fine-tuned.

When multiple candidate senses exceed the cosine similarity threshold, the system faces a tie-breaking problem. Common resolution strategies include:

• Max score selection: Choose the candidate with the highest raw cosine similarity.
• Contextual re-ranking: Use a cross-encoder model to re-score the top-k candidates with full sentence pair attention.
• Frequency prior: Favor the more common sense in the target clinical corpus (e.g., 'MI' defaults to Myocardial Infarction over Mitral Insufficiency in cardiology notes).
• Human escalation: Flag the mention for manual review when the margin between the top two scores is below a secondary delta threshold (e.g., < 0.05).