Cosine Similarity

Cosine similarity measures the cosine of the angle between two vectors, making it completely independent of their magnitudes (lengths). This is critical in text analysis, where a document's length should not affect its semantic similarity score.

• Two vectors pointing in the exact same direction yield a score of 1.0, regardless of their lengths
• Orthogonal vectors yield 0.0
• Opposite vectors yield -1.0
• Formula: cos(θ) = (A · B) / (||A|| × ||B||)

Cosine similarity excels in high-dimensional spaces where Euclidean distance suffers from the curse of dimensionality. In embedding spaces with hundreds or thousands of dimensions, the angular relationship remains a robust and computationally efficient similarity signal.

• Commonly used with 768-dimension (BERT) or 1536-dimension (Ada-002) embeddings
• Computationally dominated by the dot product, which is highly optimized on GPUs and TPUs
• Forms the backbone of vector database indexing in systems like FAISS and Annoy

For cosine similarity to function purely as an angular metric, input vectors must be L2-normalized to unit length. This transforms the calculation into a simple dot product, dramatically accelerating inference in production systems.

• Normalization step: v_normalized = v / ||v||₂
• After normalization, cos(θ) = A_normalized · B_normalized
• Many embedding models (e.g., OpenAI's text-embedding-3) output pre-normalized vectors
• Critical for metric learning with Triplet Loss, where the objective is to structure angular relationships

Cosine similarity is the scoring function at the heart of contrastive learning frameworks like SimCLR and SupCon. The model is trained to maximize cosine similarity between positive pairs (e.g., different RF captures of the same transmitter) while minimizing it for negative pairs.

• The NT-Xent loss (normalized temperature-scaled cross entropy) uses cosine similarity as its logit
• A temperature parameter τ controls the concentration of the similarity distribution
• This directly shapes the embedding space to be cosine-optimal for nearest-neighbor retrieval

Cosine similarity loses discriminative power when vectors are sparse and non-negative, such as in raw TF-IDF or count-based representations. In these cases, all vectors exist in the positive orthant, limiting the angular range to [0°, 90°] and scores to [0, 1].

• Cannot distinguish between vectors that share no overlapping features (both score 0)
• For sparse categorical data, Jaccard similarity or Hamming distance may be more appropriate
• In RF fingerprinting, dense learned embeddings avoid this sparsity problem entirely

A lower-dimensional, continuous vector space where semantically similar data points—such as RF fingerprints from the same device—are mapped close together. Cosine similarity operates directly within this space to measure angular proximity.

• Dimensionality: Typically ranges from 128 to 1024 dimensions for RF fingerprint embeddings
• Property: Distances in this space correspond to hardware similarity, not geographic proximity
• Training: Shaped by metric learning objectives to cluster same-device samples and separate different devices

A branch of machine learning focused on learning a distance function over objects. For device enrollment, metric learning trains neural networks to produce embeddings where cosine similarity reliably separates authorized from unauthorized transmitters.

• Goal: Small cosine distance between samples from the same device; large distance between different devices
• Loss Functions: Triplet loss and contrastive loss are commonly used to enforce this structure
• Outcome: Enables one-shot verification by simply thresholding the cosine similarity score

A metric-based few-shot learning architecture that computes a prototype for each class by averaging the embeddings of the support set. Classification is performed by finding the prototype with the highest cosine similarity to the query embedding.

• Enrollment: A new device prototype is computed from as few as 1-5 RF fingerprint samples
• Inference: The cosine similarity between a live signal's embedding and stored prototypes determines identity
• Advantage: Non-parametric classification; no fine-tuning required at enrollment time

A neural network architecture composed of two identical subnetworks that process pairs of inputs. The network learns to output embeddings that can be compared using cosine similarity to determine if the inputs belong to the same device.

• Training: Pairs of same-device and different-device RF samples are presented during training
• Output: The cosine similarity between the twin outputs is passed through a sigmoid to produce a similarity probability
• Use Case: Effective for one-shot enrollment where only a single reference sample is available

A probability value derived from cosine similarity indicating the model's certainty that a given RF fingerprint belongs to a specific enrolled device. This score drives authentication decisions.

• Thresholding: A minimum cosine similarity threshold (e.g., 0.95) is set to balance security and usability
• Calibration: Raw cosine values may be calibrated using Platt scaling or temperature scaling for reliable probability estimates
• Trade-off: Adjusting the threshold directly impacts False Acceptance Rate (FAR) and False Rejection Rate (FRR)

A classification paradigm where the system must identify and reject unknown devices never seen during training. Cosine similarity enables this by flagging samples that fall below a minimum similarity threshold to all known prototypes.

• Unknown Rejection: If max cosine similarity across all enrolled devices is below the threshold, the emitter is classified as 'unknown'
• Importance: Critical for detecting spoofing attacks and unauthorized devices in dynamic RF environments
• Relationship: Complements Out-of-Distribution (OOD) Detection techniques for robust security