In-Batch Negative Sampling

In a standard Two-Tower Model, a batch contains B user-item pairs. For each user, the correct item is the positive sample. The other B-1 items in the batch are treated as negative samples. The model computes the dot-product similarity for all B x B pairs simultaneously using a single matrix multiplication, making it extremely compute-efficient. The InfoNCE loss (or sampled softmax) then maximizes the score for the correct pair while minimizing scores for the in-batch negatives.

Because negatives are drawn from the empirical distribution of items in the batch, popular items appear more frequently as negatives. This causes the model to unfairly penalize popular items, pushing their embeddings away from users even when they might be relevant. The result is an embedding space where popular items are artificially depressed in ranking. Mitigation strategies include:

• LogQ correction: Adjusting logits by subtracting the log of the item's sampling probability.
• Mixed negative sampling: Combining in-batch negatives with uniformly sampled negatives.

The number of negatives per positive is B-1, making batch size a critical hyperparameter. Larger batches provide more diverse negatives, better approximating the full softmax over the entire item catalog. However, GPU memory constraints limit batch size. Typical strategies:

• Gradient accumulation across multiple micro-batches to simulate a larger effective batch.
• Cross-device batch sharing where negatives from other GPU workers are incorporated.
• Batch sizes of 2048–8192 are common in production two-tower recommenders.

A significant failure mode occurs when the batch contains multiple positive pairs for the same item. If item i is the ground-truth positive for user u, but also appears as a positive for user v in the same batch, it will be treated as a false negative for user u. This confuses the training signal. Solutions include:

• Deduplication: Ensuring each item appears only once per batch.
• Label-aware masking: Masking out in-batch negatives that are actually positives for the query user.

The temperature parameter τ controls the concentration of the similarity distribution. A low temperature (τ < 1) sharpens the distribution, making the model focus intensely on hard negatives. A high temperature (τ > 1) smooths the distribution, treating all negatives more uniformly. Temperature is typically tuned as a hyperparameter, with values between 0.05 and 0.2 being common in practice. It directly impacts the gradient magnitude flowing to hard versus easy negatives.

In-batch negative sampling is orders of magnitude faster than maintaining a separate negative sampling pool because it reuses the forward pass computations already performed for positive items. However, uniform negative sampling from the full corpus provides an unbiased estimate of the partition function. The trade-off:

• In-batch: Fast, biased (popularity skew), batch-size-dependent.
• Uniform: Slower, unbiased, requires a separate sampling infrastructure.
• Hybrid approaches combine both, using in-batch for efficiency and a small set of uniformly sampled negatives for debiasing.

The self-supervised paradigm that frames representation learning as a discrimination task. The model learns by pulling semantically similar pairs (user clicked item) closer in embedding space while pushing apart dissimilar pairs (user ignored item). The InfoNCE loss is the standard objective, where in-batch negatives serve as the contrasting samples that define the decision boundary.

A dual-encoder architecture that independently maps user features and item features into a shared embedding space. The user tower and item tower produce L2-normalized vectors, enabling efficient dot-product scoring. In-batch negative sampling is the standard training strategy for this architecture, as it reuses the item tower's outputs from the same batch as negatives, avoiding a separate negative sampling infrastructure.

An efficient training approximation that updates only a small random subset of negative item embeddings during loss computation. Instead of computing the full softmax over millions of items, the model contrasts the positive item against a handful of negatives. In-batch negative sampling is a specific variant where negatives are recycled from the current mini-batch, eliminating the need for a separate negative sampling mechanism.

The standard similarity metric in embedding-based retrieval, measuring the orientation rather than magnitude between two vectors. Calculated as the dot product of L2-normalized embeddings, it produces values in the range [-1, 1]. In two-tower training with in-batch negatives, the model maximizes cosine similarity for positive pairs while minimizing it for the in-batch negative pairs, effectively learning a metric space where relevance corresponds to angular proximity.

The retrieval infrastructure that serves embeddings at inference time. After training with in-batch negative sampling, the item tower generates embeddings for the entire catalog, which are indexed in an ANN structure like HNSW or ScaNN. User embeddings are then used to query this index for top-k retrieval. The quality of the learned embedding space directly determines ANN recall, making the training strategy critical to end-to-end system performance.

The process of constraining vectors to the unit hypersphere via L2 normalization. This transforms cosine similarity into a simple dot product, stabilizing training by bounding gradient magnitudes. In the context of in-batch negative sampling, normalization ensures that all embeddings lie on the same manifold, preventing the model from trivially minimizing loss by increasing the norm of positive embeddings rather than learning meaningful directional relationships.