Anomaly Transformer

The core innovation replacing standard self-attention. It computes two distinct attention distributions for each time point:

• Prior-Association: Models the concentration on adjacent time steps, capturing local context and short-term trends. Anomalies often disrupt this local smoothness.
• Series-Association: Models the attention to the global series, capturing long-term dependencies and dominant trends. The divergence between these two distributions forms the basis for anomaly scoring.

The primary anomaly score, replacing traditional reconstruction error. It is the symmetrized Kullback-Leibler (KL) divergence between the Prior-Association and Series-Association distributions.

• Normal Points: Exhibit high similarity between prior and series associations; the local context aligns with the global trend.
• Anomalous Points: Show a significant discrepancy. A sudden spike or dip will have a strong local prior-association but a weak series-association, resulting in a high anomaly score.

The model is trained with a specialized adversarial strategy to amplify the discrepancy for anomalies:

• Minimization Phase: The model minimizes the KL divergence for normal points, making their prior and series associations as similar as possible.
• Maximization Phase: The model maximizes the KL divergence for anomalous points, actively separating the two distributions to create a more pronounced signal. This two-phase process sharpens the model's sensitivity to deviations from normality.

Unlike autoencoders or GANs, the Anomaly Transformer does not rely on reconstructing the input signal to find errors. This provides key advantages:

• Computational Efficiency: Avoids the costly decoding step during inference.
• Superior Point-Wise Detection: Directly scores each time point based on its relational properties, rather than a holistic reconstruction loss, enabling precise localization of short anomalies within long sequences.
• Mitigates Overgeneralization: Generative models can sometimes successfully reconstruct anomalies; the association discrepancy is a more direct and robust signal.

The model's output is refined through a two-step inference process:

1. Raw Discrepancy Calculation: The association discrepancy is computed for every time point in the sequence.
2. Dynamic Thresholding & Pruning: A threshold is applied to generate a candidate set of anomalous points. A secondary step then prunes false positives by analyzing the temporal continuity of the candidates, ensuring that only genuine anomalous segments are flagged.

The foundational computational unit of the Anomaly Transformer that models two distinct attention distributions simultaneously. The prior-association branch captures local temporal concentration by attending strongly to adjacent time steps, representing short-term patterns. The series-association branch learns global trends by attending across the entire sequence. The association discrepancy—the symmetric KL divergence between these two distributions—serves as the anomaly criterion. Points where local and global attention diverge significantly are flagged as anomalous, as they represent events that break from expected temporal dynamics.

The core anomaly scoring metric computed as the symmetric Kullback-Leibler divergence between the prior-association and series-association distributions at each time point. A high discrepancy indicates that a point's local temporal context conflicts with the global sequence pattern. This metric is minimax optimized during training: the model amplifies the discrepancy for anomalous points while minimizing it for normal points, creating a sharp separation. At inference, the association discrepancy directly produces the anomaly score without requiring reconstruction error or density estimation.

A specialized training procedure that treats anomaly detection as a two-player adversarial game within a single model. Phase one minimizes the association discrepancy for normal points, training the attention to align local and global views on expected behavior. Phase two maximizes the discrepancy for synthetic or identified anomalies, forcing the model to amplify the signal at irregular time steps. This minimax strategy prevents the model from learning a trivial uniform attention pattern and ensures the discrepancy metric remains a reliable discriminator between normal and anomalous subsequences.

A competing architecture for time-series modeling that uses dilated causal convolutions instead of attention mechanisms. TCNs capture long-range dependencies through exponentially increasing dilation factors while maintaining computational efficiency. Unlike the Anomaly Transformer, TCNs lack an explicit discrepancy mechanism and typically rely on reconstruction error or forecasting error for anomaly scoring. They serve as a strong baseline, particularly for tasks where local receptive fields and translation invariance are more important than global attention patterns.

A transformer variant designed explicitly for long sequence time-series forecasting that introduces ProbSparse self-attention to reduce quadratic complexity. The Informer selects only dominant query-key pairs based on a sparsity measurement, dramatically reducing memory usage. While optimized for forecasting rather than anomaly detection, its efficient attention mechanism has influenced anomaly transformer designs. The key distinction is that Informer focuses on predictive accuracy, whereas the Anomaly Transformer uses attention distributions themselves as the detection signal.

The algorithmic task of finding subsequences that are maximally dissimilar to all other subsequences in a time series. Matrix Profile is the canonical technique, computing the z-normalized Euclidean distance from each subsequence to its nearest non-self match. The Anomaly Transformer generalizes this concept by learning a learnable dissimilarity metric through attention discrepancy rather than relying on fixed distance functions. This allows it to detect semantic anomalies that may appear numerically similar but structurally inconsistent with learned temporal patterns.