Prediction Interval

A prediction interval estimates the range for a single future observation, while a confidence interval estimates the range for an unknown population parameter (like the mean). Prediction intervals are inherently wider because they must account for both the uncertainty in estimating the model's parameters and the irreducible random variability of the individual data point itself.

The defining property is the nominal coverage probability, typically set at 90%, 95%, or 99%. A well-calibrated 95% prediction interval should contain the actual realized value 95% of the time over many repeated trials. Key metrics for evaluating this property include:

• Prediction Interval Coverage Probability (PICP): The empirical percentage of observations falling within the interval.
• Sharpness: The average width of the interval, where narrower is better, assuming correct coverage.

A modern, non-parametric approach to building intervals is through quantile regression. Instead of assuming a Gaussian distribution, a model is trained to directly predict specific percentiles of the target variable using the pinball loss function. A 90% prediction interval is formed by taking the difference between the predicted 95th quantile (upper bound) and the 5th quantile (lower bound). This method makes no assumptions about the shape of the forecast distribution.

Classical statistical models like ARIMA and deep learning models like DeepAR construct intervals by assuming the forecast errors follow a specific probability distribution, most commonly the Gaussian (Normal) distribution. The interval is then calculated as the point forecast plus/minus a critical value multiplied by the forecast standard deviation. This method is efficient but can be misleading if the assumption of normality is violated by heavy-tailed or asymmetric data.

A distribution-free, model-agnostic technique that provides rigorous mathematical guarantees of coverage without assuming any underlying data distribution. Inductive Conformal Prediction works by:

• Holding out a calibration dataset not used in model training.
• Computing nonconformity scores (absolute residuals) on this set.
• Using the empirical distribution of these scores to construct intervals for new points that are guaranteed to contain the true value at the specified confidence level, assuming data exchangeability.

For complex models that produce a full predictive distribution, intervals are derived through simulation. A model might output the parameters of a negative binomial or Student's t-distribution. By drawing thousands of random samples from this predicted distribution, the empirical quantiles of the simulated paths define the prediction interval. This is the standard output of models like Temporal Fusion Transformer (TFT) and is evaluated using the Continuous Ranked Probability Score (CRPS).

The foundational paradigm that outputs a full probability distribution of possible future outcomes rather than a single point estimate. Unlike deterministic methods that give a false sense of precision, probabilistic forecasting enables risk-aware decision-making by explicitly modeling the aleatoric uncertainty inherent in demand patterns. Modern implementations use deep autoregressive models like DeepAR to output parametric distributions (Gaussian, negative binomial) at each time step.

A statistical technique that directly estimates specific percentiles of the conditional distribution without assuming a parametric form. By training separate models for the 5th and 95th percentiles, you construct asymmetric prediction intervals that capture heteroscedastic uncertainty—where variance changes with the input features. The key innovation is using pinball loss as the objective function, which asymmetrically penalizes over-prediction and under-prediction based on the target quantile.

A strictly proper scoring rule that evaluates the entire predicted cumulative distribution function against the observed outcome. CRPS generalizes the Mean Absolute Error to probabilistic forecasts—it reduces to MAE when the forecast is deterministic. Lower CRPS values indicate better calibration and sharpness simultaneously. It is the gold-standard metric for comparing probabilistic models because it rewards forecasts that are both confident and correct.

The direct operational application of prediction intervals in inventory management. Safety stock is calculated as: Z × σ_LT, where Z is the z-score corresponding to the desired service level (e.g., 1.65 for 95%), and σ_LT is the standard deviation of forecast error over the lead time. By using the upper bound of a prediction interval instead of a point forecast, supply chain planners explicitly buffer against demand variability and avoid costly stockouts.

A distribution-free, model-agnostic framework for constructing prediction intervals with finite-sample validity guarantees. Unlike traditional methods that assume Gaussian errors, conformal prediction wraps around any black-box model and uses a held-out calibration set to determine interval widths. The guarantee: for a confidence level of 1-α, the true value falls within the interval at least 1-α of the time, regardless of the underlying distribution.

The systematic tendency of a model to consistently over-predict or under-predict, measured as the mean of forecast errors over time. A well-calibrated prediction interval should contain the actual value with the nominal coverage probability—if a 90% interval captures only 70% of outcomes, the model exhibits under-dispersion bias and is overconfident. Tracking realized coverage against nominal coverage is essential for monitoring prediction interval health in production.