Temporal SHAP

Temporal SHAP is built on cooperative game theory, where each time step is treated as a player in a coalition. The method calculates the marginal contribution of a time step by averaging its impact across all possible subsets of other time steps. This guarantees three critical properties:

• Local Accuracy: The sum of all time-step attributions equals the model's prediction difference from a baseline
• Missingness: A time step with no impact receives zero attribution
• Consistency: If a model changes so a time step's contribution increases, its attribution never decreases

The result is a unique, fair distribution of credit that no other attribution method can simultaneously satisfy.

Exact Shapley value computation is exponentially complex with sequence length. Temporal SHAP employs optimized sampling to make the calculation tractable:

• Permutation Sampling: Randomly orders time steps and computes incremental contributions, averaging over many permutations
• Paired Sampling: Uses antithetic pairs to reduce variance in the Monte Carlo estimate
• Stratified Sampling: Ensures balanced representation of coalition sizes across the sampling distribution
• Dynamic Stopping: Monitors convergence of attribution estimates and halts sampling when values stabilize

These strategies enable practical application to sequences with hundreds of time steps while maintaining statistical rigor.

The choice of baseline or reference value critically shapes Temporal SHAP explanations. Common strategies include:

• Zero Baseline: Replaces masked time steps with zeros, suitable for normalized data
• Mean Imputation: Uses the feature-wise mean of the training distribution
• Segmented Baseline: Draws from a distribution of realistic values within the same seasonal or trend segment
• Counterfactual Baseline: Uses a specific alternative scenario, such as a previous period's values

The baseline represents the absence of information from a time step. Selecting a contextually appropriate baseline is essential for generating explanations that align with domain expectations.

Temporal SHAP extends beyond simple time-step importance to decompose attributions across the feature-time matrix. This reveals interactions such as:

• A specific sensor reading at a specific lag driving a forecast
• The combined effect of multiple features within a critical time window
• Temporal interactions where the importance of a time step depends on values at preceding steps

This granular decomposition is achieved by treating each feature-time tuple as an independent player in the Shapley game, enabling engineers to pinpoint exactly which measurements at which moments drive model behavior.

Temporal SHAP is model-agnostic and applies to diverse sequence architectures:

• LSTM/GRU Networks: Explains which time steps the recurrent state retains or forgets
• Temporal Convolutional Networks: Attributes importance across the receptive field of dilated convolutions
• Transformer Models: Decomposes the influence of positional encodings and self-attention across the sequence
• Temporal Fusion Transformer: Complements the architecture's native variable selection with time-step granularity

Implementation typically wraps the model with a prediction function that accepts masked inputs, enabling SHAP to query the model under different temporal coalitions without modifying internal architecture.

Rigorous validation ensures Temporal SHAP explanations are trustworthy:

• Faithfulness Correlation: Measures whether time steps with higher attribution cause larger prediction changes when perturbed
• Stability Index: Quantifies explanation consistency across similar input sequences
• Sensitivity Analysis: Tests attribution robustness to small input perturbations and sampling noise
• Ablation Sanity Checks: Verifies that removing top-attributed time steps degrades performance more than removing low-ranked steps

These metrics distinguish genuine temporal drivers from artifacts of the explanation method, critical for high-stakes applications in finance and healthcare where explanations inform decisions.