Time-Series Forecasting

Adapt convolutional neural networks (CNNs) for sequential data by using causal, dilated convolutions.

• Causal Convolutions: Ensure an output at time t is convolved only with elements from time t and earlier, preventing information leakage from the future.
• Dilated Convolutions: Allow the network to have an exponentially large receptive field with a limited number of layers by skipping inputs at a defined stride.
• Advantages: TCNs can process sequences in parallel (unlike RNNs), exhibit stable gradients, and can handle very long sequences efficiently. They are a strong alternative to RNNs for many forecasting tasks.

Architectures that use self-attention mechanisms to model dependencies across all time steps in a sequence, regardless of distance.

• Self-Attention: Computes a weighted sum of all past values, where weights are determined by the compatibility between the current query and past keys. This allows direct modeling of long-range dependencies.
• Positional Encoding: Injects information about the relative or absolute position of time steps in the sequence, as the attention mechanism itself is permutation-invariant.
• Examples: Models like Informer, Autoformer, and Temporal Fusion Transformer (TFT) are specifically designed for long-horizon time-series forecasting, often incorporating probabilistic outputs and interpretable attention patterns.

Models that predict a probability distribution over future values, rather than a single point estimate, quantifying uncertainty.

• Output: Typically produces prediction intervals (e.g., 95% confidence interval) or full parametric distributions (e.g., Gaussian, Negative Binomial).
• Key Methods:
  • Quantile Regression: Models specific percentiles of the target distribution.
  • DeepAR: An autoregressive RNN-based model that outputs parameters of a chosen distribution (likelihood) for the next time step.
  • Conformal Prediction: A post-hoc method that uses past forecast errors to calibrate prediction intervals from any underlying model, providing distribution-free guarantees. Probabilistic forecasts are critical for risk-aware decision-making in fields like finance, supply chain, and energy.

Combining multiple forecasting models to improve accuracy and robustness beyond any single model's capability.

• Model Averaging: Simple averaging or weighted averaging of predictions from diverse models (e.g., ARIMA, ETS, ML model).
• Stacking (Meta-Learning): Using predictions from multiple base models as features to train a final "meta-model" (like linear regression or a simple MLP) that learns how to best combine them.
• Hybrid Models: Architecturally integrating different model types. For example, using a CNN or TCN to extract local features and an LSTM to model long-term dependencies in a single network. Ensembles reduce variance and mitigate model-specific biases, often leading to top performance in forecasting competitions.

The broader machine learning task of forecasting the next element(s) in an ordered series. While time-series forecasting is a specific case using regular time intervals, sequence prediction includes any ordered data.

• Models Used: Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, Gated Recurrent Units (GRUs), and Transformer architectures with causal attention masks.
• Applications: Next-word prediction in language models, forecasting stock prices, predicting the next frame in a video, or anticipating a user's next action in a software interface.

A continuous, time-ordered sequence of discrete events or state changes. This is the raw material for many forecasting problems in agentic systems, where events may be irregularly spaced.

• Characteristics: Each event has a timestamp and a payload (e.g., {timestamp: 2024-05-15T10:30:00Z, event: 'user_login', user_id: 123}).
• Processing: Streams are often processed by frameworks like Apache Kafka or Apache Flink to perform real-time aggregations, detect patterns, and generate features for forecasting models.
• Forecasting Challenge: Predicting the timing and type of future events, such as server failures or transaction fraud.

An operation in Convolutional Neural Networks (CNNs) where filters are applied across the time dimension to extract local temporal patterns from sequential data. Temporal Convolutional Networks (TCNs) use dilated convolutions to capture long-range dependencies with fewer layers.

• Mechanism: A filter slides over the sequence, performing element-wise multiplication and summation to produce a feature map highlighting local trends or shapes.
• Advantage: Can be more computationally efficient and easier to parallelize than recurrent models for certain forecasting tasks.
• Use Case: Anomaly detection in sensor data, audio waveform processing, and action recognition in video.

A mechanism within neural networks that dynamically weights the importance of past observations when making a prediction for the current time step. It allows a model to focus on relevant historical periods, regardless of their distance.

• In Transformers: The decoder uses a causal (masked) self-attention layer to attend only to previous elements in the sequence, preventing information leakage from the future.
• Benefit: Overcomes the limitation of fixed-size context windows in RNNs and can directly model long-term dependencies. Models like the Informer and Autoformer use specialized attention mechanisms for efficient long-sequence time-series forecasting.
• Agentic Relevance: Enables an agent to selectively recall which past experiences are most pertinent to its current decision.