Time-Series Saliency Map

The canonical representation is a heatmap overlaid directly on the input time series, where color intensity encodes the magnitude of importance. Warmer colors (red/orange) indicate high saliency, while cooler colors (blue) indicate low relevance. This allows practitioners to instantly identify critical temporal regions—such as a sudden spike preceding a failure prediction or a seasonal peak driving a demand forecast. The overlay preserves the original signal morphology, enabling direct visual correlation between input patterns and attribution scores.

For multi-dimensional time series, saliency maps extend to 2D heatmaps where rows represent features (sensors, asset prices, vital signs) and columns represent time steps. This matrix visualization reveals cross-feature temporal dynamics:

• Which sensor lagged behind others in signaling an anomaly
• How leading indicators in one channel influence lagging responses in another
• Whether the model correctly attends to known causal relationships (e.g., heart rate before blood pressure change) This format is essential for validating models in industrial IoT and clinical monitoring.

A perturbation-based method that generates saliency maps by systematically sliding a masking window across the input sequence. At each position, the occluded segment is replaced with a baseline (zero, mean, or noise), and the change in model output is measured. The resulting occlusion curve directly quantifies the prediction's sensitivity to each temporal interval. Unlike gradient-based methods, occlusion analysis is model-agnostic—it works identically on LSTMs, Transformers, and even black-box forecasting APIs—making it the universal fallback for interpretability.

For differentiable models, saliency is computed by taking the gradient of the output with respect to each input time step. The magnitude of the gradient indicates how much a small perturbation at that step would change the prediction. Key variants include:

• Vanilla Gradient: Raw input gradients, often noisy
• Integrated Gradients: Accumulates gradients along a path from a baseline, satisfying the completeness axiom where attributions sum to the prediction difference
• SmoothGrad: Averages gradients over noisy copies of the input to reduce visual noise These methods provide fine-grained, instantaneous importance scores.

In Transformer-based time-series models (e.g., Temporal Fusion Transformer, Informer), attention weights provide a built-in saliency mechanism. The attention matrix—where each cell represents the weight from a query time step to a key time step—can be visualized as a temporal dependency graph. Self-attention patterns reveal:

• Long-range dependencies where distant historical events influence current predictions
• Sparse attention patterns indicating the model has learned to ignore irrelevant history
• Multi-head diversity showing different heads capturing different temporal scales (short-term vs. seasonal) Attention-based saliency is computationally free since weights are already computed during inference.

Not all saliency maps accurately reflect the model's true reasoning. Faithfulness metrics quantitatively assess explanation quality:

• Comprehensiveness: Removing the top-k salient time steps should cause a large prediction drop
• Sufficiency: Keeping only the top-k salient steps should preserve the prediction
• Monotonicity: Prediction change should correlate monotonically with cumulative saliency removal
• Continuity: Similar inputs should produce similar saliency maps These metrics prevent reliance on plausible-looking but misleading visualizations, ensuring the saliency map is a trustworthy audit artifact.