Self-Supervised Anomaly Detection

The core mechanism involves training a model on a surrogate task where labels are automatically generated from the data itself. Common pretext tasks include:

• Masked prediction: Reconstructing intentionally corrupted or held-out features
• Contrastive learning: Pulling augmented views of the same instance together while pushing apart different instances
• Jigsaw puzzle solving: Predicting the correct spatial or temporal ordering of shuffled patches or subsequences
• Rotation prediction: Identifying the applied geometric transformation to an input The model learns rich representations of normality by solving these tasks on unlabeled data.

Anomalies are detected by their failure to conform to the learned structure. Scoring approaches include:

• Pretext task error: High error on the surrogate task (e.g., poor reconstruction of a masked transaction feature) indicates an outlier
• Representation deviation: Distance from the centroid of normal embeddings in the learned latent space
• Consistency violation: Inconsistency across multiple augmented views of the same instance, where normal data produces stable predictions This paradigm avoids the need for labeled anomalies during training.

Contrastive methods learn an embedding space where normal transactions cluster tightly and anomalies are pushed to low-density regions. The process:

• Positive pairs: Two augmented views of the same normal transaction (e.g., with noise added)
• Negative pairs: Different transactions assumed to be dissimilar
• Loss function: Typically InfoNCE or NT-Xent loss, maximizing mutual information between positive pairs At inference, the anomaly score is the distance to the nearest cluster centroid or the local density in the embedding space.

Inspired by masked language modeling in NLP, this approach randomly masks a subset of transaction features and trains a model to predict the masked values from the visible context. For financial data:

• Input masking: Randomly zero out or replace features like amount, merchant category, or timestamp
• Reconstruction target: Predict the original values of masked features
• Anomaly signal: Fraudulent transactions exhibit higher prediction error because their feature combinations violate the learned joint distribution of normal behavior This method naturally handles mixed categorical and numerical features.

For sequential transaction data, self-supervised tasks exploit temporal structure to learn normal behavioral patterns:

• Next-event prediction: Forecast the next transaction attributes given the history
• Sequence order verification: Determine if a subsequence is in the correct temporal order
• Interpolation consistency: Predict intermediate states between two observed transactions Anomalous sequences break these temporal regularities, producing high surprisal scores that flag potential fraud campaigns or account takeover patterns.

Self-supervised approaches offer distinct benefits compared to traditional unsupervised anomaly detection:

• Richer representations: Learn semantically meaningful features rather than relying on raw distances or density estimates
• Reduced assumptions: No need to assume a specific distribution (unlike GMM) or density threshold (unlike LOF)
• Adaptability: Pretext tasks can be tailored to domain-specific fraud patterns
• Label efficiency: Leverages massive unlabeled transaction volumes while providing a stronger signal than purely unsupervised methods
• Transfer potential: Learned representations can be fine-tuned if labeled anomalies become available later.