Anomaly Detection

These methods assume that the normal data points are generated by an underlying statistical distribution. Anomalies are defined as observations that have a low probability of being generated by this learned model.

• Gaussian Mixture Models (GMMs): Fit multiple Gaussian distributions to the data and flag points in low-density regions.
• Box Plot Rule: A simple, non-parametric heuristic where points falling outside 1.5 times the interquartile range (IQR) are flagged.
• Grubbs' Test: A formal statistical test used to detect a single outlier in a univariate dataset that follows an approximate normal distribution.

These algorithms operate on the premise that normal data points occur in dense neighborhoods, while outliers are far from their nearest neighbors.

• K-Nearest Neighbors (KNN): The anomaly score is the distance to the k-th nearest neighbor. Points with a large distance are considered anomalous.
• Local Outlier Factor (LOF): A density-based method that measures the local deviation of a data point's density relative to its neighbors, effectively identifying points in substantially lower-density regions.
• Cluster-Based Local Outlier Factor (CBLOF): Performs clustering first and then calculates an outlier score based on the size of the cluster and the distance to the nearest large cluster centroid.

These methods explicitly isolate anomalies instead of profiling normal points. They exploit the fact that anomalies are few and different, making them easier to separate.

• Isolation Forest: Builds an ensemble of random trees where splits are made on randomly selected features and split values. The path length to isolate a point is averaged; anomalies have distinctly shorter paths.
• Extended Isolation Forest: An enhancement that uses hyperplanes with random slopes for splits, overcoming the bias toward axis-parallel artifacts in the standard algorithm.
• SCiForest: A variant that uses a sliding window to select features, improving performance on high-dimensional data with many irrelevant attributes.

Neural network architectures designed to learn a compact representation of normality and flag deviations in the learned latent space.

• Deep SVDD (Support Vector Data Description): Trains a neural network to map normal data into a minimal hypersphere centered at a fixed point. The distance to the center serves as the anomaly score.
• Autoencoder Reconstruction Error: A bottleneck architecture compresses and reconstructs data. Anomalies are identified by a high reconstruction error, as the model has only learned to faithfully reconstruct normal patterns.
• Variational Autoencoder (VAE): Uses a probabilistic latent space. Anomaly scoring can leverage the reconstruction probability, which is more principled than raw reconstruction error.

Techniques designed to find anomalies that are only visible in specific lower-dimensional projections of the data, often hidden in high-dimensional noise.

• Principal Component Analysis (PCA): Anomalies are detected by high reconstruction error when projecting data onto a lower-dimensional hyperplane defined by the principal components.
• Robust PCA: Decomposes a data matrix into a low-rank component (normal structure) and a sparse component (anomalies), making it highly effective for detecting outliers in high-dimensional spaces.
• High-Contrast Subspaces: Methods that search for specific subspaces where the density contrast between a point and its neighbors is maximized, revealing anomalies masked in the full feature space.

Specialized algorithms for sequential data that account for temporal dependencies, trends, and seasonality.

• ARIMA Residual Analysis: Fits an autoregressive integrated moving average model and flags points where the residual error exceeds a dynamic threshold.
• STL Decomposition: Decomposes a time series into seasonal, trend, and residual components using LOESS smoothing. Anomalies are detected in the residual component.
• Spectral Residual (SR): A fast unsupervised method that analyzes the spectral residual in the frequency domain to detect anomalies in real-time streaming data.
• LSTM/Transformer Forecasting: Deep learning models predict the next value in a sequence. A significant deviation between the predicted and actual value triggers an anomaly alert.

The task of identifying inputs that are semantically or statistically different from a model's training data. While anomaly detection often focuses on rare in-distribution events, OOD detection specifically flags inputs from a fundamentally different distribution.

• Prevents unpredictable predictions on unknown concepts
• Critical for safety-critical ML deployments
• Uses methods like Energy-Based Models and Mahalanobis Distance

An ensemble-based algorithm that isolates observations by randomly selecting a feature and split value. The core insight: anomalies are few and different, requiring fewer random splits to isolate.

• Path length from root to leaf serves as the anomaly score
• Linear time complexity with low memory footprint
• Effective for high-dimensional data without distance calculations

A density-based method measuring the local deviation of a data point's density relative to its k-nearest neighbors. Points in substantially lower-density regions are flagged as anomalies.

• Captures local density variations that global methods miss
• Effective for datasets with clusters of varying densities
• LOF score ~1 indicates inlier; significantly >1 indicates outlier

Two fundamental types of uncertainty critical for anomaly detection:

• Epistemic Uncertainty: Model uncertainty from lack of knowledge, reducible with more data. High in unfamiliar regions of input space.
• Aleatoric Uncertainty: Irreducible noise inherent in the data itself, such as sensor error or class overlap.

Distinguishing between them enables more precise anomaly flagging.

A training strategy that leverages an auxiliary dataset of outliers to teach the model heuristics for detecting unknown inputs. Rather than relying solely on in-distribution data, the model learns explicit decision boundaries against anomalous examples.

• Significantly improves generalization to unseen OOD distributions
• Can use synthetic or naturally occurring outlier datasets
• Reduces false positives on challenging near-OOD inputs

Deep One-Class Classification that trains a neural network to map normal data into a minimal hypersphere. Points far from the sphere center are flagged as anomalies.

• Extends Support Vector Data Description with deep feature learning
• Soft-boundary variant handles contaminated training data
• Jointly optimizes feature representation and anomaly boundary