Temporal Prediction Difference

The method works by replacing the original value at time step t with a non-informative baseline (often the dataset mean or a zero vector) and observing the change in the model's prediction score.

• Prediction Difference: Importance(x_t) = f(x) - E[f(x | x_t = baseline)]
• This directly answers: How much does knowing the actual value at this moment change the model's mind?
• Unlike gradient methods, this captures non-linear saturation effects where large input changes cause negligible output shifts.

A naive approach sets the target time step to zero, but this creates out-of-distribution artifacts. Temporal Prediction Difference uses proper marginalization:

• Single Reference: Replaces the value with a static baseline like the training set mean.
• Conditional Expectation: Samples replacement values from the conditional distribution p(x_t | x_{/t}) to preserve realistic temporal dynamics.
• Multiple Imputation: Averages over many plausible replacement values to reduce variance, yielding a more statistically robust importance estimate.

In LSTMs and GRUs, the hidden state h_t acts as a memory bottleneck. Marginalizing an early time step reveals its long-range impact:

• Vanishing Influence Detection: If marginalizing x_1 causes no prediction change, the model failed to learn a long-term dependency.
• Gate Interaction Analysis: The method exposes how the forget gate at a specific step erases prior context.
• Anomaly Localization: In industrial IoT, engineers use this to pinpoint the exact sensor reading that triggered a failure forecast.

Exact marginalization requires integrating over all possible values, which is intractable. Practical implementations use approximation strategies:

• Monte Carlo Sampling: Draw N samples from the baseline distribution and average the output difference.
• Dynamic Reference Windows: For streaming data, use a rolling window of recent history as the baseline distribution.
• Complexity: O(N * T) forward passes for a sequence of length T, making it more expensive than gradient-based methods but more faithful for non-linear models.

While both methods attribute importance to time steps, they differ fundamentally in their approach:

• Temporal Prediction Difference: A perturbation-based method that measures output change from a single baseline replacement.
• Temporal Integrated Gradients: A gradient-based method that integrates gradients along a linear path from baseline to input.
• Saturation Sensitivity: Prediction Difference correctly assigns zero importance to saturated features, whereas Integrated Gradients may still attribute non-zero scores due to the integral path.
• Axiom Compliance: Prediction Difference does not satisfy the completeness axiom, but it is more intuitive for counterfactual reasoning.

The method has known vulnerabilities that practitioners must account for:

• Baseline Dependency: Importance scores are highly sensitive to the choice of baseline distribution. A poor baseline yields misleading attributions.
• Interaction Blindness: Marginalizing one step at a time misses epistatic interactions where two time steps jointly influence the output.
• Out-of-Distribution Sampling: If the conditional expectation model is weak, replacement values create unrealistic sequences that trigger model artifacts.
• High Variance: Monte Carlo approximations require many samples for stable estimates, limiting real-time applicability.

Adapts Shapley value calculations from cooperative game theory to assign importance scores to individual time steps. It treats each time step as a player in a coalition, computing the marginal contribution of a step by averaging its effect across all possible subsets of other steps. This provides a axiomatically fair distribution of credit for a sequence model's prediction, satisfying properties like efficiency and symmetry.

A perturbation-based method that systematically removes or masks individual time steps from a sequence to measure the resulting change in model output. By replacing a step's value with a neutral baseline (zero, mean, or noise) and observing the prediction delta, engineers can rank steps by importance. This technique is model-agnostic and computationally straightforward but may miss interaction effects between steps.

A gradient-based attribution technique that computes the integral of gradients along a straight-line path from a baseline input to the actual input. Adapted for time-series data, it satisfies the sensitivity axiom: if changing a time step changes the prediction, that step receives non-zero attribution. This method is particularly effective for deep learning models where gradients are readily available via backpropagation.

Tracks the propagation of attention weights across multiple layers in a Transformer model. By multiplying attention matrices and accounting for residual connections, it quantifies how information from a specific time step cascades through the network to influence the final prediction. This reveals the effective receptive field of each step, distinguishing between direct and indirect attention pathways.

Generates a minimally altered time-series path that would cause a forecasting model to produce a different, desired outcome. Unlike feature attribution, which explains why a prediction occurred, counterfactuals answer what-if questions by identifying the smallest realistic perturbation needed to flip a classification or change a forecast. This is critical for recourse and scenario planning in financial and operational contexts.

Slides a masking window across a time series, occluding contiguous segments to generate a saliency map. By measuring the prediction change when each interval is hidden, it reveals which temporal intervals are most critical. Unlike single-step ablation, this method captures the importance of multi-step patterns and subsequences, making it ideal for identifying event-driven dependencies in sensor data and IoT streams.