Temporal Fusion Transformer (TFT)

TFT does not assume all input features are relevant. At each time step, it uses dedicated gating mechanisms to select the most salient static and temporal variables.

• Static Covariate Encoders: Learn which time-invariant features (e.g., product category, store location) are relevant.
• Temporal Variable Selection: Dynamically weights past observations and known future inputs, discarding noisy predictors.
• This prevents irrelevant data from dominating the attention layers, improving generalization on noisy retail datasets.

The primary building block of the TFT, the GRN, controls the flow of non-linear information through the model.

• Gating Mechanism: Uses a sigmoid gate to suppress unnecessary components, allowing the network to adapt its depth dynamically.
• Residual Connection: Ensures stable gradient flow during training by adding the input directly to the output.
• ELU Activation: Employs Exponential Linear Units for faster convergence.
• This architecture allows TFT to handle both simple linear relationships and complex non-linear interactions without overfitting.

TFT modifies standard transformers to interpret long-range temporal dependencies explicitly.

• Interpretable Multi-Head Attention: Shares values across heads but uses distinct query/key weights, allowing the model to learn different temporal patterns (e.g., weekly vs. monthly seasonality).
• Attention Masks: Ensures causality by preventing future time steps from leaking into the past.
• Long-Range Dependencies: Unlike RNNs, the attention mechanism can directly link a current demand spike to a promotion that occurred 30 days ago.

TFT is natively a probabilistic forecasting model. It outputs predictions at specific quantiles (e.g., P10, P50, P90) rather than a single point estimate.

• Pinball Loss: The model is trained to minimize the pinball loss across all desired quantiles simultaneously.
• Uncertainty Quantification: The spread between the P10 and P90 outputs forms a prediction interval, allowing supply chain directors to set safety stock based on risk tolerance.
• This is critical for inventory optimization where the cost of a stockout far exceeds the cost of overstock.

TFT integrates static metadata directly into the temporal processing flow, a key differentiator from pure time-series models.

• Static Enrichment Layer: Encoded static features (e.g., store size, product margin) are injected into the temporal feature vectors.
• Context Vectors: Static encoders produce context vectors that prime the temporal attention and gating mechanisms.
• Cold-Start Handling: This allows the model to generate reasonable forecasts for a new product launch by leveraging its static attributes, even without sales history.

TFT uses a standard encoder-decoder architecture optimized for time series.

• LSTM Encoder: Processes the historical context (past sales, prices) to generate a compressed representation of the past.
• LSTM Decoder: Generates the multi-horizon forecast, attending to the encoder output.
• Look-Ahead Bias Prevention: The decoder uses only known future inputs (e.g., planned promotions, calendar events) and the encoder's context, never peeking at future actuals.
• This structure naturally handles the heterogeneous inputs common in retail demand forecasting.

A pure deep learning architecture that does not use any time-series-specific feature engineering. It decomposes a signal into trend and seasonality using a backward and forward residual link structure.

• Key innovation: Basis expansion layers that learn polynomial and harmonic functions
• Contrast with TFT: N-BEATS is univariate and produces point forecasts; TFT is multivariate, natively handles static covariates, and outputs probabilistic quantiles
• Best for: Short-term, high-volume univariate series where interpretability is not required

An autoregressive recurrent neural network that learns a global model across hundreds of related time series. It outputs a parametric distribution (e.g., negative binomial for count data) at each time step.

• Key innovation: Shared parameters across items enable cold-start forecasting for products with no history
• Contrast with TFT: DeepAR uses LSTM recurrence; TFT uses self-attention, which captures long-range dependencies more effectively and provides variable selection weights for interpretability
• Best for: Large-scale product catalogs with sparse or intermittent demand

Instead of a single point estimate, the model outputs a full predictive distribution—typically quantiles like P10, P50, and P90. This enables risk-aware inventory decisions.

• TFT advantage: Uses quantile regression with pinball loss to directly output multiple quantiles without assuming a parametric distribution
• Business value: A P90 forecast tells you the demand level you'll cover 90% of the time, directly informing safety stock calculations
• Key metric: Continuous Ranked Probability Score (CRPS) evaluates the entire distribution against the observed outcome

A core TFT component that learns which input features are relevant at each time step, suppressing noisy or irrelevant variables. This is critical for demand forecasting where hundreds of covariates may be available.

• Static covariate encoders: Process time-invariant features like product category or store location
• Temporal variable selection: Applies attention to past observations and known future inputs (e.g., planned promotions, holidays)
• Interpretability output: Feature importance weights are directly extractable, unlike black-box RNNs

Forecasts generated at SKU, category, and regional levels must be mathematically coherent—the sum of lower-level forecasts should equal the aggregate forecast.

• TFT integration: TFT produces base forecasts at any level; reconciliation methods like MinT (Minimum Trace) optimally adjust them to enforce coherence
• Top-down: Forecast at the top level, then disaggregate using historical proportions
• Bottom-up: Forecast at the most granular level, then sum upward
• Middle-out: Combines both approaches for the best accuracy at all levels

The multi-head self-attention in TFT allows the model to learn long-range temporal dependencies without the vanishing gradient problems of RNNs. Each head can attend to different patterns simultaneously.

• Interpretable multi-head attention: Unlike standard transformers, TFT modifies the attention mechanism to share values across heads and use additive aggregation, producing a single set of attention weights
• Seasonality capture: Attention heads can specialize in weekly, monthly, or quarterly patterns
• Contrast with LSTMs: Attention provides direct access to any past time step, not just the compressed hidden state