One-Class SVM

The algorithm's objective is to find a hyperplane (or hypersphere in the kernel space) that maximizes the margin between the origin and the normal data points. It treats the origin as a synthetic outlier and learns a decision function that returns +1 for data within the learned region and -1 for points outside. This is formalized by solving a quadratic programming problem to minimize the volume of the hypersphere while capturing most training data.

• Key Parameter: nu controls the upper bound on the fraction of training errors (outliers) and the lower bound on the number of support vectors.
• Mathematical Form: It finds a weight vector w and offset ρ such that most data points satisfy w·Φ(x_i) ≥ ρ - ξ_i, where ξ_i are slack variables.

Like standard SVMs, One-Class SVM employs the kernel trick to project data into a higher-dimensional feature space without explicit computation. This allows it to learn a flexible, non-linear boundary around the target class.

• Common Kernels: The Radial Basis Function (RBF) kernel is most prevalent, enabling the model to fit complex, irregular shapes around normal data.
• Implicit Mapping: The kernel function K(x_i, x_j) = Φ(x_i)·Φ(x_j) computes similarities, letting the algorithm operate in a rich feature space where a simple hyperplane can separate normal data from the origin effectively.

One-Class SVM is fundamentally a semi-supervised algorithm. It requires only examples of the normal or target class during training. It does not need labeled anomalous examples, making it ideal for scenarios where anomalies are rare, costly to label, or unknown.

• Training Data: The model is fit solely on data assumed to be normal.
• Use Case Fit: Perfect for novelty detection in machinery (normal operation only), fraud (legitimate transactions are abundant), or system monitoring (baseline healthy state).

The model outputs a signed distance from the learned boundary. A positive value indicates the sample is inside the normal region; a negative value flags it as an outlier. This is a score, not a calibrated probability.

• Thresholding: A threshold (often zero) is applied to this score to make a binary normal/abnormal classification. This threshold can be adjusted to control the false positive rate.
• Interpretation: The magnitude of the score indicates the degree of 'normality' or 'abnormality,' with more negative scores being stronger outliers.

One-Class SVM vs. Other Anomaly Detectors:

• vs. Isolation Forest: Isolation Forest explicitly isolates anomalies, often faster on large datasets, but One-Class SVM can provide a more precise, margin-based boundary with kernels.
• vs. Local Outlier Factor (LOF): LOF is density-based and local. One-Class SVM learns a global description of normality, which can be more robust in data with varying densities.
• vs. Autoencoders: Autoencoders learn to reconstruct normal data; anomalies have high reconstruction error. One-Class SVM provides a geometric boundary, often requiring less data and being less prone to overfitting on small normal datasets.

Effective deployment requires careful parameter selection and validation.

• Critical Parameter nu: Expected outlier fraction. Sets a trade-off between the model's tightness and tolerance for misclassifying training points.
• Kernel Parameter gamma (for RBF): Controls the influence radius of a single training point. Low gamma = broad influence, smoother boundary. High gamma = tight influence, complex boundary that can overfit.
• Scalability: Training involves a quadratic programming problem, making it computationally intensive for very large datasets (>10k-50k samples).
• Validation Challenge: Without labeled anomalies, use techniques like hold-out validation on presumed-normal data or synthetic contamination.

The overarching machine learning task of identifying new or previously unseen patterns in data. One-Class SVM is a canonical algorithm for this task. Key distinctions:

• Training Data: Typically uses only 'normal' data.
• Goal: To learn a model of normality and flag any significant deviation.
• Contrast with Outlier Detection: Often used synonymously, but novelty detection implies the 'anomalies' are new types of events not represented during training.

A closely related algorithm to One-Class SVM. Instead of finding a maximal margin hyperplane, SVDD finds a minimal hypersphere that encloses the target data in a high-dimensional feature space. Points outside the sphere are classified as anomalies.

• Mathematical Formulation: Minimizes the volume of the sphere subject to constraints.
• Kernel Trick: Like SVM, uses kernels to handle non-linear data boundaries.
• Use Case: Particularly effective when the normal data is assumed to be concentrated in a single cluster.

An ensemble, tree-based algorithm for unsupervised anomaly detection. It explicitly isolates anomalies rather than profiling normal points.

• Mechanism: Anomalies are 'few and different,' so they require fewer random partitions in a tree structure to be isolated.
• Anomaly Score: Derived from the average path length from the root to the leaf node across all trees. Shorter paths indicate higher anomaly likelihood.
• Advantages: Efficient on large datasets, handles high dimensionality well, and does not rely on distance or density measures.

A density-based algorithm that identifies outliers by measuring the local deviation of a data point's density compared to its neighbors.

• Core Concept: Calculates how isolated a point is with respect to the surrounding neighborhood.
• Key Metric: The LOF score. A score approximately equal to 1 indicates density similar to neighbors. A score significantly greater than 1 indicates an outlier.
• Contrast with One-Class SVM: LOF is purely unsupervised and can identify outliers within a dataset, whereas One-Class SVM learns a boundary from only normal data to classify new points.

An unsupervised deep learning approach where a neural network is trained to compress and reconstruct normal data.

• Process: An autoencoder learns a compressed representation (encoding) of the input data and then tries to reconstruct it. It is trained to minimize reconstruction error on normal data.
• Anomaly Signal: During inference, data with a high reconstruction error is flagged as anomalous, as the model cannot accurately reproduce patterns it hasn't learned.
• Comparison: Unlike the discriminative boundary of One-Class SVM, this is a generative model-based approach. It can capture complex, non-linear manifolds of normal data.

A multivariate distance metric used for outlier detection that accounts for correlations between dataset features.

• Formula: Distance = √[(x - μ)ᵀ S⁻¹ (x - μ)], where x is the observation, μ is the mean vector, and S is the covariance matrix.
• Intuition: Measures how many standard deviations a point is away from the mean of a distribution, considering the shape of the distribution.
• Application: Often used to define a threshold (e.g., points beyond 3 standard deviations) for flagging multivariate outliers. It assumes the data follows a multivariate Gaussian distribution, a stricter assumption than the non-parametric One-Class SVM.