Triplet Loss

The loss function operates on three distinct inputs simultaneously:

• Anchor (a): The reference sample (e.g., a specific user's current session).
• Positive (p): A sample belonging to the same class or context as the anchor (e.g., an item the user clicked).
• Negative (n): A sample from a different class (e.g., an item the user ignored or a random non-interaction). The goal is to learn an embedding function f(x) such that the distance between the anchor and the positive is strictly less than the distance between the anchor and the negative.

A critical hyperparameter that defines the minimum separation enforced between positive and negative pairs:

• The loss is zero only if distance(a, n) > distance(a, p) + α.
• Hard margin: Enforces a strict boundary, which can be sensitive to outliers.
• Soft margin: Uses a smooth approximation (e.g., log(1 + exp(d_ap - d_an))) for more stable gradient flow.
• Typical values range from 0.1 to 1.0, controlling how tightly clusters form and how far apart dissimilar classes are pushed.

The quality of learned embeddings depends heavily on the selection of triplets during training:

• Random triplets: Easy to generate but produce weak gradients because the constraint is already satisfied.
• Semi-hard negatives: Negatives that are farther than the positive but still within the margin. These provide the most informative gradients.
• Hard negatives: Negatives that are closer to the anchor than the positive. These violate the constraint and drive strong corrective updates.
• Hard positive mining: Selecting positives that are visually or semantically dissimilar to the anchor improves robustness to intra-class variance.

The standard triplet loss is defined as: L = max(0, d(a, p) - d(a, n) + α) where d is typically the Euclidean distance or cosine distance.

• The max(0, ·) function ensures the loss is active only when the margin constraint is violated.
• When the negative is sufficiently far, the loss contribution is zero, preventing the network from needlessly collapsing embeddings.
• Batch-hard strategies compute the loss using only the hardest positive and hardest negative within each mini-batch for efficient training.

Triplet loss directly optimizes the relative ordering of items for user embedding generation:

• User-to-Item: Anchor is a user embedding, positive is a clicked item, negative is a skipped or random item.
• Session-based: Anchor is a partial session, positive is the next-clicked item, negative is a random catalog item.
• Cross-modal: Aligning text descriptions with product images by treating matching pairs as positives and mismatched pairs as negatives. This relative ranking objective often outperforms pointwise losses like binary cross-entropy for top-N recommendation tasks.

While both are metric learning objectives, they differ in their constraints:

• Contrastive Loss: Operates on pairs and minimizes distance for similar pairs while maximizing it for dissimilar pairs up to a margin. It uses an absolute distance threshold.
• Triplet Loss: Operates on triplets and enforces a relative distance constraint. It does not require defining an absolute threshold for similarity.
• Triplet loss is generally more flexible for ranking tasks because it cares about the ordering of distances rather than their absolute values, making it well-suited for retrieval and recommendation.

The performance of Triplet Loss is highly sensitive to the composition of the input tuple. The anchor is the reference sample, the positive shares the same class, and the negative belongs to a different class. Effective training relies on hard negative mining, where the negative is chosen because it is closer to the anchor than the positive, violating the margin constraint and providing a strong gradient signal. Random or easy negatives produce near-zero loss and stall learning.