Metric Learning

Unlike traditional algorithms that use fixed distance measures, metric learning parameterizes the distance function itself. The model learns a mapping to an embedding space where distances correspond to semantic similarity.

• Mahalanobis distance is a common parameterized form: d(x,y) = sqrt((x-y)ᵀ M (x-y)) where M is learned.
• Deep metric learning uses neural networks to learn highly non-linear distance functions.
• The learned metric is task-specific, optimized for the exact definition of similarity required by the application.

The core output of metric learning is a structured latent space where geometric relationships have semantic meaning. This space is organized by the loss function to enforce specific properties.

• Intra-class compactness: Samples from the same class are pulled together into tight clusters.
• Inter-class separability: Clusters of different classes are pushed apart, often with a margin.
• The resulting space supports efficient nearest neighbor search for identification and retrieval tasks.

Metric learning is operationalized through specific loss functions that define how embeddings should be organized. The two foundational formulations are contrastive loss and triplet loss.

• Contrastive loss operates on pairs, minimizing distance for positive pairs and enforcing a margin for negative pairs.
• Triplet loss uses an anchor, a positive, and a negative sample, ensuring the anchor is closer to the positive than the negative by a margin α.
• Advanced formulations like N-pair loss and lifted structure loss leverage multiple negatives simultaneously for faster convergence.

A critical advantage of metric learning is its ability to generalize to open-set recognition problems. The model learns a similarity function rather than a fixed set of class boundaries.

• In zero-shot scenarios, embeddings for unseen classes can be computed and compared without retraining.
• This is essential for emitter identification where new devices must be enrolled dynamically.
• The model outputs a similarity score, allowing a threshold to decide if a sample belongs to any known class or is entirely novel.

The quality of the learned metric depends heavily on the training examples. Hard negative mining selects the most informative, difficult samples to accelerate learning.

• A hard negative is a sample from a different class that is incorrectly closer to the anchor than some positives.
• Training exclusively on easy negatives yields weak decision boundaries.
• Modern approaches use semi-hard mining, selecting negatives within a margin band, to balance training stability and embedding quality.

To overcome the computational cost of pairwise or triplet sampling, proxy-based methods learn a small set of representative embeddings for each class.

• A proxy is a learnable vector that represents an entire class in the embedding space.
• Losses like Proxy-NCA and Proxy-Anchor compare samples to proxies instead of other samples, reducing complexity from O(N²) to O(N).
• This approach is highly scalable for large datasets and is widely used in deep metric learning for retrieval.

A loss function that operates on triplets of samples: an anchor, a positive (same device), and a negative (different device). The objective is to learn an embedding where the distance between the anchor and the positive is less than the distance to the negative by a specified margin.

• Hard negative mining is critical: selecting negatives that are close to the anchor but from a different class drives discriminative learning.
• Prevents mode collapse by explicitly enforcing relative distance constraints rather than absolute positions.
• Commonly paired with Siamese networks for one-shot and few-shot device enrollment scenarios.

A neural architecture composed of two or more identical subnetworks that share the same weights and parameters. Each subnetwork processes a distinct input, and their outputs are fed into a distance metric layer to compute similarity.

• Weight sharing guarantees that identical inputs produce identical embeddings, a fundamental requirement for consistent device comparison.
• During inference, the subnetwork acts as a standalone feature extractor, generating embeddings for a lookup database.
• Often trained with contrastive loss or triplet loss to minimize intra-class distance and maximize inter-class distance.

A self-supervised paradigm that learns representations by comparing pairs of data points. The model is trained to pull positive pairs (augmentations of the same signal) together and push negative pairs (different signals) apart in the latent space.

• Frameworks like SimCLR and MoCo use large batch sizes or momentum queues to provide diverse negative examples.
• Eliminates the need for labeled device identities during pre-training; the model learns channel-invariant features directly from raw IQ data.
• The learned representations transfer effectively to downstream tasks like open set recognition and device authentication.

The mathematical function that quantifies similarity between two learned embeddings. The choice of metric directly impacts clustering behavior and nearest-neighbor retrieval accuracy.

• Euclidean distance (L2): Standard for triplet loss; assumes a spherical embedding space.
• Cosine similarity: Measures angular distance, robust to magnitude variations caused by channel gain differences.
• Mahalanobis distance: Accounts for feature correlations and variance, useful when embedding dimensions are not independent.
• Learnable metrics: A small neural network can be trained to compute a non-linear similarity score, replacing fixed mathematical functions.

An alternative to pair-based losses that assigns a learnable proxy vector to each class. The loss optimizes the distance between samples and their corresponding proxies rather than between individual sample pairs.

• Proxy-NCA and Proxy Anchor Loss dramatically reduce training complexity from O(N²) to O(N×C), where C is the number of classes.
• Proxies act as class centroids, stabilizing training and converging faster than triplet-based methods.
• Particularly effective for large-scale device identification where exhaustive pair sampling is computationally prohibitive.

A family of loss functions that enforce angular separability in the embedding space by adding an additive or multiplicative margin to the decision boundary between classes.

• ArcFace, CosFace, and SphereFace normalize both weights and features to lie on a hypersphere, making the loss sensitive only to angular distance.
• The margin penalty explicitly forces a gap between the closest samples of different classes, improving open set rejection capabilities.
• Widely adopted in face recognition and adapted for RF fingerprinting to create highly discriminative, channel-robust embeddings.