PatchTST

Instead of attending to individual time steps, PatchTST groups consecutive steps into subseries-level patches.

• Local semantic capture: Each patch encodes a short-term pattern, such as a micro-trend or a bid-ask bounce cycle.
• Reduced computational cost: By attending to patches rather than raw ticks, the quadratic complexity of self-attention is slashed. A sequence of 512 ticks can be reduced to 32 patches.
• Extended receptive field: The model can see much longer historical windows, crucial for capturing slow-burn market regimes while still processing tick-level data.

Example: A 64-tick window of LOB updates is segmented into 4 patches of length 16, each representing a distinct phase of order book pressure.

PatchTST treats each univariate time series independently, sharing the same model weights across all channels but not mixing information between them.

• Avoids noise cross-contamination: In high-frequency finance, a spurious correlation between order flow and a random tick can poison a mixed-channel model. Channel-independence prevents this.
• Simplifies feature engineering: Each predictor (e.g., bid-ask spread, OFI, VWAP) is forecast independently, making the signal cleaner and easier to interpret.
• Proven superior accuracy: Empirical results show channel-independence outperforms channel-mixing on standard benchmarks, a critical advantage for noisy market microstructure data.

Example: A model forecasts the next 100 ticks of bid-ask spread using only its own history, without being distracted by simultaneous volume spikes.

Once the time series is segmented into patches, a vanilla Transformer encoder applies multi-head self-attention over the patch representations.

• Global pattern recognition: The attention mechanism learns long-range dependencies between patches, such as how an early order book imbalance patch relates to a later volatility spike patch.
• Parallel computation: Unlike recurrent models (LSTMs), the Transformer processes all patches simultaneously, enabling faster training on large tick history datasets.
• No positional information loss: Learned positional embeddings are added to each patch to preserve the temporal order of market events.

Example: The model attends strongly from a patch showing a sudden OFI surge to a patch 50 steps later showing a price jump, learning the causal lag.

PatchTST applies reversible instance normalization to combat the distribution shift between training and test periods—a chronic problem in financial data.

• Zero-mean, unit-variance normalization: Each input window is normalized independently before patching, removing non-stationary trends like a slow drift in average spread size.
• Reversal for output: The normalization statistics are reapplied to the model's output, restoring the original scale of the forecast.
• Robust to regime change: This technique makes the model agnostic to the absolute level of a variable, focusing purely on the shape and pattern of the time series.

Example: A model trained on a low-volatility month generalizes to a high-volatility earnings day because the input is normalized to a standard scale before patching.

PatchTST can be pre-trained using a masked autoencoder objective, where random patches are hidden and the model must reconstruct them.

• Leverages unlabeled tick data: Vast archives of historical LOB data with no specific trading signal can be used to pre-train a model that learns the universal grammar of market microstructure.
• Superior fine-tuning: A pre-trained PatchTST model adapts to a specific forecasting task (e.g., predicting mid-price moves) with far fewer labeled samples, reducing the risk of backtest overfitting.
• Transfer learning across assets: A model pre-trained on liquid equity order books can be fine-tuned for a less liquid ETF, transferring learned microstructure patterns.

Example: A model is pre-trained on 5 years of raw LOB data for AAPL by reconstructing masked patches, then fine-tuned on 6 months of labeled triple barrier outcomes.

The final patch representations are flattened and passed through a simple linear layer to produce the forecast.

• Point forecasts: A linear head directly outputs the predicted values for the horizon.
• Probabilistic outputs: The head can be configured to output the parameters of a distribution (e.g., mean and variance for a Gaussian), enabling conformal prediction intervals and risk quantification.
• Simplicity prevents overfitting: A lightweight linear head ensures the heavy lifting is done by the Transformer encoder, preventing the decoder from memorizing noise in the training set.

Example: The model outputs a 95% prediction interval for the VWAP over the next 100 ticks, allowing an execution algorithm to dynamically adjust its urgency based on forecast uncertainty.

A core design principle of PatchTST where each univariate series in a multivariate dataset is processed by a shared weight transformer backbone independently. This contrasts with channel-mixing models that blend information across variables at each time step.

• Mechanism: The model treats a multivariate time series with C channels as C separate univariate series.
• Advantage: Prevents the model from overfitting to spurious cross-channel correlations in noisy financial data.
• Result: Superior performance on datasets where individual channel dynamics are more predictive than inter-channel relationships.

The process of segmenting a long input time series into overlapping or non-overlapping subseries-level patches before feeding them into the transformer. This is analogous to tokenization in NLP but for continuous time-series data.

• Patch Length (P): The number of time steps in each patch, controlling the granularity of local semantics.
• Stride (S): The step size between consecutive patches, controlling the degree of overlap.
• Benefit: Reduces the effective sequence length from L to roughly L/S, slashing the quadratic complexity of self-attention and allowing the model to handle very long look-back windows efficiently.

PatchTST leverages the multi-head self-attention mechanism to capture global dependencies across the entire historical window. By patching first, the model can attend to semantic units (patches) rather than individual noisy ticks.

• Receptive Field: The patching mechanism, combined with the transformer's global attention, allows the model to learn dependencies spanning the entire look-back window.
• Contrast with TCNs: Unlike Temporal Convolutional Networks, which have a finite receptive field determined by kernel size and dilation, PatchTST can directly relate a patch at the beginning of the series to one at the end.
• Financial Application: Captures long-range seasonal patterns and regime shifts in high-frequency data without manual feature engineering.

A preprocessing technique applied to each input window before patching to mitigate the distribution shift between training and testing data. It normalizes each input sequence to have zero mean and unit standard deviation.

• Reversible: The normalization statistics are applied inversely to the model's output to restore the original scale.
• Purpose: Addresses the common time-series problem where the mean and variance of a financial series change over time, causing train-test discrepancy.
• Integration: Often used as a simple wrapper around the PatchTST backbone to improve robustness to non-stationary price data.

PatchTST is architected for effective self-supervised representation learning through masked patch prediction, but its core forecasting model is trained with supervised loss. The patching structure is inherently compatible with pre-training.

• Masked Autoencoder (MAE) Style: Random patches are masked, and the model learns to reconstruct them from the visible context.
• Transfer Learning: Pre-trained weights on a large corpus of financial data can be fine-tuned on a specific asset with limited history.
• Loss Function: Typically trained with Mean Squared Error (MSE) loss on the forecast horizon.

While the base PatchTST model produces a point forecast (a single predicted value per time step), the architecture can be extended to output a probabilistic distribution. This is critical for risk management in quantitative finance.

• Distribution Head: The final linear layer can be replaced with a head that outputs the parameters of a chosen distribution (e.g., Gaussian mean and variance, or Student's-t parameters).
• Loss Function: Trained by minimizing the Negative Log-Likelihood (NLL) of the observed value under the predicted distribution.
• Output: Generates prediction intervals and Value-at-Risk (VaR) estimates essential for algorithmic trading risk controls.