One-Class SVM

Unlike binary SVM, One-Class SVM treats the origin of the feature space as the sole representative of the anomaly class. The algorithm constructs a maximum-margin hyperplane that separates the mapped training data from the origin with the widest possible margin. This geometric trick allows the model to learn a compact boundary around normal data without requiring labeled fraud examples.

• The ν (nu) parameter controls the fraction of training points allowed to fall between the hyperplane and the origin
• Points on the wrong side of the decision boundary are classified as anomalies
• This formulation is equivalent to finding the minimum-volume hypersphere enclosing the data in kernel space

One-Class SVM projects data into a high-dimensional reproducing kernel Hilbert space where complex, non-linear decision boundaries become linear. The Radial Basis Function kernel is the most common choice for financial fraud detection, as it can model the intricate, non-linear manifolds of legitimate transaction behavior.

• RBF kernel: Captures local density variations; parameter γ controls the radius of influence of each support vector
• Polynomial kernel: Models higher-order feature interactions for structured deviation patterns
• Linear kernel: Used when the feature space is already high-dimensional or interpretability is critical
• The kernel choice directly impacts the model's ability to separate subtle fraud patterns from normal variance

The ν parameter is the central tuning knob of One-Class SVM, simultaneously controlling two critical properties. It sets an upper bound on the fraction of margin errors and a lower bound on the fraction of support vectors.

• ν ∈ (0, 1] defines the expected outlier proportion in the training set
• A ν of 0.01 means the model expects approximately 1% contamination
• Lower ν values create tighter boundaries, increasing sensitivity but risking overfitting
• Higher ν values produce looser boundaries, trading false negatives for false positives
• In fraud detection, ν is typically set conservatively low (0.001–0.05) to reflect the rarity of known fraud

The final model is sparse: only the training points lying on or within the margin—the support vectors—define the decision function. All other points are discarded, making inference computationally efficient.

• Support vectors are the critical normal instances that shape the boundary
• Points far inside the boundary contribute zero weight to the decision function
• The number of support vectors grows with ν and the complexity of the data distribution
• A high support vector ratio indicates a complex, potentially overfit boundary
• This sparsity is a key advantage over density-based methods like Kernel Density Estimation for large-scale scoring

New transactions are scored using the signed distance to the separating hyperplane in kernel space. The decision function returns a real-valued score where negative values indicate anomalies and positive values indicate normal instances.

• f(x) = sign(Σ αᵢ K(xᵢ, x) - ρ) where ρ is the offset and αᵢ are dual coefficients
• The magnitude of the score reflects confidence: more negative = more anomalous
• Scores can be calibrated into probability estimates using Platt scaling or isotonic regression
• Threshold tuning on a validation set is essential to balance precision and recall for operational deployment
• Unlike Isolation Forest, the score is a continuous distance metric, not a path length

One-Class SVM assumes the training data is predominantly normal with minimal contamination. It is sensitive to outliers in the training set, which can distort the learned boundary. The algorithm also scales poorly to massive datasets due to its quadratic memory complexity.

• Training complexity: O(n²) to O(n³) depending on the solver; impractical beyond ~100,000 samples without approximations
• Sensitivity to scaling: Features must be standardized; the RBF kernel is not scale-invariant
• Contamination robustness: Even 1-2% anomalies in training can significantly warp the boundary
• For large-scale financial data, consider Stochastic Gradient Descent One-Class SVM or Isolation Forest as alternatives
• Works best when normal behavior is stationary and well-sampled