DAGMM

Unlike two-stage methods that first reduce dimensionality and then fit a density estimator, DAGMM simultaneously trains a deep autoencoder and a Gaussian Mixture Model (GMM). The compression network learns a low-dimensional latent representation, while the estimation network evaluates the likelihood of that representation under the GMM. This end-to-end training ensures the latent space is explicitly structured to be GMM-friendly, maximizing the density contrast between normal data and anomalies.

The estimation network is a feed-forward neural network that takes the latent representation and the reconstruction error features as input. It outputs a soft mixture membership prediction for each sample. The model computes the sample energy—the negative log-likelihood of a sample under the fitted GMM—which serves directly as the anomaly score. High energy indicates a low-density region, flagging the point as an outlier.

DAGMM does not use a single scalar reconstruction error. Instead, it computes a multi-dimensional error vector including:

• Euclidean distance between input and output
• Cosine similarity between input and output
• Element-wise relative error statistics This rich error representation provides the estimation network with granular information about how a sample deviates from the learned manifold, improving anomaly discrimination.

Direct joint training can converge to poor local minima where the autoencoder maps all data to a trivial latent space. DAGMM mitigates this with a hierarchical pre-training strategy: the autoencoder is first trained independently to reconstruct normal data, and the GMM is initialized on the resulting latent representations. This provides a stable starting point before the full end-to-end fine-tuning begins.

During GMM fitting, covariance matrices can collapse to singularities, causing infinite likelihoods. DAGMM introduces a regularization penalty that penalizes extremely small eigenvalues in the covariance matrices. This keeps the GMM numerically stable and prevents the model from overfitting to spurious high-density regions in the latent space.

Once trained, DAGMM provides an anomaly score in a single forward pass without requiring iterative optimization. The input is fed through the autoencoder to get the latent code and error features, then through the estimation network to compute the mixture parameters and sample energy. This makes DAGMM suitable for batch anomaly detection on large financial transaction datasets.

The probabilistic foundation of DAGMM's estimation network. A GMM assumes data is generated from a mixture of k Gaussian distributions. Unlike k-means hard clustering, GMM provides a soft probability density for each point. In DAGMM, the GMM is fitted in the low-dimensional latent space, and the sample energy (negative log-likelihood) serves as the anomaly score. Low-density regions correspond to anomalies.

A critical sub-network in DAGMM that replaces the traditional Expectation-Maximization (EM) algorithm. It takes the low-dimensional latent representation and the reconstruction error features as input, then predicts the mixture membership for each sample. This allows the entire model—compression and density estimation—to be trained end-to-end with stochastic gradient descent, avoiding the instability of alternating optimization.

DAGMM explicitly concatenates derived reconstruction error features with the latent code before density estimation. These features include:

• Relative Euclidean distance: ||x - x'||₂ / ||x||₂
• Cosine similarity: cos(x, x') These metrics capture the multi-scale deviation from normality, preventing the compression network from discarding critical anomaly signals that might not be preserved in the latent bottleneck alone.

Unlike two-stage methods (e.g., train an autoencoder, then fit a separate GMM), DAGMM optimizes a combined loss function:

• Reconstruction Loss: L(x, x') ensures the autoencoder learns the manifold of normal data.
• Energy Loss: The negative log-likelihood from the GMM penalizes high energy (low density) in the latent space. This joint optimization prevents the autoencoder from learning a latent space that is suboptimal for density estimation.

A competing deep anomaly detection method that maps normal data into a minimal hypersphere in the latent space. Unlike DAGMM's probabilistic mixture approach, Deep SVDD uses a geometric center as the reference. Anomalies are scored by their distance from the hypersphere center. DAGMM offers an advantage in complex, multi-modal normal patterns where a single hypersphere is insufficient to capture the distribution.

The final output metric derived from the GMM. The sample energy E(z) = -log( Σₖ φ̂ₖ N(z|μ̂ₖ, Σ̂ₖ) ) measures how likely a compressed sample is under the learned mixture density. High energy indicates a point in a sparse region of the latent space. A threshold on this energy is used to classify transactions as fraudulent, providing a principled, likelihood-based score rather than a heuristic distance.