Contrastive Learning

The core mechanism relies on constructing positive pairs (augmented views of the same instance or semantically similar samples) and negative pairs (dissimilar instances). In fraud detection, a legitimate transaction and its perturbed version form a positive pair, while a fraudulent transaction serves as a negative. The model learns to pull positives together and push negatives apart in the embedding space.

The Noise Contrastive Estimation loss, or InfoNCE, is the standard objective. It treats the task as a categorical classification problem where the model must identify the positive sample among a set of negative distractors. The loss minimizes the negative log-likelihood of correctly classifying the positive pair, encouraging a low-temperature, highly discriminative embedding space.

Instead of manual labels, contrastive learning uses stochastic data augmentation to create positive pairs. For tabular financial data, augmentations include:

• Feature masking: randomly zeroing out transaction fields
• Gaussian noise injection: adding calibrated jitter to numerical amounts
• Swap noise: shuffling categorical features within a permissible range These corruptions force the encoder to learn invariances to non-fraudulent variations.

The architecture typically uses an encoder network (e.g., a ResNet or Transformer) followed by a projection head—a small multi-layer perceptron. Contrastive loss is applied to the projection head's output, not the encoder's. After training, the projection head is discarded, and the frozen encoder produces high-quality representations where anomaly scores can be computed via distance to normal cluster centroids.

Random negative samples are often too easy to discriminate, leading to saturated loss and poor gradients. Hard negative mining strategically selects negative samples that are close to the anchor in the current embedding space but are semantically different. In fraud contexts, this means selecting legitimate transactions that superficially resemble fraud, forcing the model to learn subtle, discriminative features.

Two foundational frameworks define modern contrastive learning:

• SimCLR: Uses large batch sizes to provide many negative examples simultaneously, relying on aggressive data augmentation and a non-linear projection head.
• MoCo: Maintains a dynamic dictionary of negative representations via a momentum encoder and a queue, decoupling the dictionary size from the batch size for more stable, compute-efficient training on financial datasets.

The fundamental mechanism of contrastive learning. Positive pairs are semantically similar samples (e.g., two augmentations of the same transaction, or a user's historical behavior). Negative pairs are dissimilar samples (e.g., different users' transactions). The model is trained to minimize the distance between positive pairs and maximize the distance between negative pairs in the embedding space. In fraud detection, defining negatives is critical: random sampling often yields easy negatives, while hard negative mining selects transactions that are close to the anchor but belong to a different class, significantly improving the model's discriminative power.

The standard objective function for contrastive learning, also known as Noise Contrastive Estimation. It treats the task as a categorical classification problem where the model must identify the correct positive pair among a set of negative distractors. Mathematically, it maximizes the mutual information between representations. Key properties:

• Uses a temperature parameter (τ) to control the concentration of the distribution; lower values sharpen the penalty on hard negatives.
• Scales efficiently with large batch sizes, which provide more negative samples.
• Forms the theoretical backbone of frameworks like SimCLR and MoCo.

A key architectural innovation from the Momentum Contrast (MoCo) framework that decouples the dictionary size from the batch size. Instead of using the same encoder for both query and key representations, MoCo maintains a slowly progressing key encoder updated via exponential moving average (EMA) of the query encoder's weights. This produces a consistent, large, and dynamic dictionary of negative representations stored in a queue. Benefits for fraud detection:

• Enables learning from a vast number of negative transaction patterns without requiring enormous GPU memory.
• The momentum update prevents representation collapse, where all outputs map to a trivial constant vector.

A simple framework for contrastive learning of visual representations, adapted for tabular and sequence data. SimCLR relies on large batch sizes instead of a momentum encoder or memory bank. Its core components:

• Stochastic data augmentation: Generates two correlated views of the same input (e.g., masking, noise injection, permutation).
• Projection head: A small MLP (g(·)) that maps representations to a space where contrastive loss is applied, preventing the encoder from discarding information useful for downstream anomaly scoring.
• The architecture demonstrates that the composition of augmentations and the inclusion of a non-linear projection head are critical for learning separable representations.

A classic metric learning loss function that predates modern contrastive frameworks. It operates on triplets: an anchor (a), a positive (p), and a negative (n). The objective is to ensure the distance between anchor and negative is greater than the distance between anchor and positive by a specified margin (α). Formally: L = max(d(a,p) - d(a,n) + α, 0). In anomaly detection, triplets can be constructed using known normal transactions as anchors, slightly perturbed versions as positives, and known fraudulent or out-of-distribution samples as negatives, explicitly enforcing a margin of separation.

An extension of self-supervised contrastive learning that leverages label information to create more structured embeddings. Unlike cross-entropy, which only pulls samples of the same class together indirectly, Supervised Contrastive Loss explicitly pulls all samples belonging to the same class (e.g., 'fraud') closer while pushing all samples from different classes apart. This results in:

• Embedding spaces where clusters are more compact and well-separated.
• Improved robustness to noisy labels compared to cross-entropy.
• Direct applicability to fraud datasets where labeled anomalies exist, allowing the model to learn a representation that generalizes to unseen fraud types.