Time-Series Forecasting

These foundational models assume specific structures in the data, such as trends and seasonality.

• ARIMA (AutoRegressive Integrated Moving Average): Models a series using its own past values (autoregression), past forecast errors (moving average), and differencing to make it stationary. The order is defined as ARIMA(p,d,q).
• Exponential Smoothing (ETS): Applies exponentially decreasing weights to past observations, with variations like Holt-Winters to capture trend and seasonality.
• Prophet: An additive model developed by Meta, designed for business time series with strong seasonal effects and holidays. It decomposes a series into trend, seasonality, and holiday components. These models are interpretable and effective for univariate series with clear patterns but struggle with high-dimensional or highly non-linear data.

A class of neural networks designed for sequential data, where connections form a directed cycle, allowing information to persist.

• Core Mechanism: The network maintains a hidden state that acts as a memory of previous inputs in the sequence.
• Long Short-Term Memory (LSTM): Introduces gating mechanisms (input, forget, output gates) to control the flow of information, effectively solving the vanishing gradient problem and capturing long-range dependencies.
• Gated Recurrent Unit (GRU): A simplified variant of LSTM with a reset and update gate, offering similar performance with fewer parameters. RNNs are powerful for sequence-to-sequence tasks but can be computationally intensive to train and are inherently sequential, limiting parallelization.

Adapt convolutional neural networks (CNNs) for sequential data by using causal, dilated convolutions.

• Causal Convolutions: Ensure an output at time t is convolved only with elements from time t and earlier, preventing information leakage from the future.
• Dilated Convolutions: Allow the network to have an exponentially large receptive field with a limited number of layers by skipping inputs at a defined stride.
• Advantages: TCNs can process sequences in parallel (unlike RNNs), exhibit stable gradients, and can handle very long sequences efficiently. They are a strong alternative to RNNs for many forecasting tasks.

Architectures that use self-attention mechanisms to model dependencies across all time steps in a sequence, regardless of distance.

• Self-Attention: Computes a weighted sum of all past values, where weights are determined by the compatibility between the current query and past keys. This allows direct modeling of long-range dependencies.
• Positional Encoding: Injects information about the relative or absolute position of time steps in the sequence, as the attention mechanism itself is permutation-invariant.
• Examples: Models like Informer, Autoformer, and Temporal Fusion Transformer (TFT) are specifically designed for long-horizon time-series forecasting, often incorporating probabilistic outputs and interpretable attention patterns.

Models that predict a probability distribution over future values, rather than a single point estimate, quantifying uncertainty.

• Output: Typically produces prediction intervals (e.g., 95% confidence interval) or full parametric distributions (e.g., Gaussian, Negative Binomial).
• Key Methods:
  • Quantile Regression: Models specific percentiles of the target distribution.
  • DeepAR: An autoregressive RNN-based model that outputs parameters of a chosen distribution (likelihood) for the next time step.
  • Conformal Prediction: A post-hoc method that uses past forecast errors to calibrate prediction intervals from any underlying model, providing distribution-free guarantees. Probabilistic forecasts are critical for risk-aware decision-making in fields like finance, supply chain, and energy.

Combining multiple forecasting models to improve accuracy and robustness beyond any single model's capability.

• Model Averaging: Simple averaging or weighted averaging of predictions from diverse models (e.g., ARIMA, ETS, ML model).
• Stacking (Meta-Learning): Using predictions from multiple base models as features to train a final "meta-model" (like linear regression or a simple MLP) that learns how to best combine them.
• Hybrid Models: Architecturally integrating different model types. For example, using a CNN or TCN to extract local features and an LSTM to model long-term dependencies in a single network. Ensembles reduce variance and mitigate model-specific biases, often leading to top performance in forecasting competitions.

A database system optimized for storing, querying, and analyzing sequences of time-stamped data points. Unlike traditional relational databases, TSDBs like InfluxDB or TimescaleDB are engineered for high write throughput, efficient time-range queries, and automatic data retention policies. They are the foundational storage layer for telemetry, metrics, and event streams that feed forecasting models.

• Key Features: High ingestion rates, built-in time-based aggregation functions, and downsampling for long-term data retention.
• Use Case: Storing sensor readings, application logs, or financial tick data for real-time monitoring and historical analysis.

A statistical relationship where the value or occurrence of an event at one time influences values at future times. This is the fundamental assumption of time-series forecasting. Dependencies can be:

• Autocorrelation: The correlation of a signal with a lagged version of itself (e.g., today's temperature is similar to yesterday's).
• Seasonality: Regular, predictable patterns that repeat over fixed periods (e.g., daily, weekly, yearly cycles in retail sales).
• Trend: A long-term, underlying direction of the data (e.g., gradual increase in website traffic).

Models like ARIMA (AutoRegressive Integrated Moving Average) explicitly model these dependencies to generate predictions.

The broader machine learning task of forecasting the next element(s) in an ordered series. While time-series forecasting is a specific case using regular time intervals, sequence prediction includes any ordered data.

• Models Used: Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, Gated Recurrent Units (GRUs), and Transformer architectures with causal attention masks.
• Applications: Next-word prediction in language models, forecasting stock prices, predicting the next frame in a video, or anticipating a user's next action in a software interface.

A continuous, time-ordered sequence of discrete events or state changes. This is the raw material for many forecasting problems in agentic systems, where events may be irregularly spaced.

• Characteristics: Each event has a timestamp and a payload (e.g., {timestamp: 2024-05-15T10:30:00Z, event: 'user_login', user_id: 123}).
• Processing: Streams are often processed by frameworks like Apache Kafka or Apache Flink to perform real-time aggregations, detect patterns, and generate features for forecasting models.
• Forecasting Challenge: Predicting the timing and type of future events, such as server failures or transaction fraud.

An operation in Convolutional Neural Networks (CNNs) where filters are applied across the time dimension to extract local temporal patterns from sequential data. Temporal Convolutional Networks (TCNs) use dilated convolutions to capture long-range dependencies with fewer layers.

• Mechanism: A filter slides over the sequence, performing element-wise multiplication and summation to produce a feature map highlighting local trends or shapes.
• Advantage: Can be more computationally efficient and easier to parallelize than recurrent models for certain forecasting tasks.
• Use Case: Anomaly detection in sensor data, audio waveform processing, and action recognition in video.

A mechanism within neural networks that dynamically weights the importance of past observations when making a prediction for the current time step. It allows a model to focus on relevant historical periods, regardless of their distance.

• In Transformers: The decoder uses a causal (masked) self-attention layer to attend only to previous elements in the sequence, preventing information leakage from the future.
• Benefit: Overcomes the limitation of fixed-size context windows in RNNs and can directly model long-term dependencies. Models like the Informer and Autoformer use specialized attention mechanisms for efficient long-sequence time-series forecasting.
• Agentic Relevance: Enables an agent to selectively recall which past experiences are most pertinent to its current decision.