Continuous Ranked Probability Score (CRPS)

The practice of generating a full predictive probability distribution over future outcomes rather than a single point estimate. This approach quantifies uncertainty, enabling risk-aware inventory decisions. CRPS is the standard metric for evaluating these forecasts because it assesses the entire distribution's alignment with the observed outcome, rewarding both sharpness (concentration) and calibration (statistical consistency).

A range derived from a forecast distribution within which a future observation is expected to fall with a specified nominal coverage probability (e.g., 90%). While a prediction interval provides a simple uncertainty summary, CRPS evaluates the full distribution, penalizing forecasts that produce intervals that are either too wide (uninformative) or too narrow (overconfident) relative to the observed outcome.

An asymmetric scoring function used to train and evaluate quantile regression models. For a target quantile τ, it penalizes over-prediction by a factor of τ and under-prediction by (1-τ). CRPS can be mathematically expressed as the integral of the pinball loss over all quantiles from 0 to 1, making it a continuous generalization that evaluates the entire distribution simultaneously.

A statistical technique that estimates specific conditional quantiles of the response variable. In demand forecasting, this enables asymmetric prediction intervals (e.g., a 95th percentile for safety stock). CRPS provides a unified summary of performance across all quantiles, avoiding the need to select and evaluate arbitrary quantile levels individually.

The systematic tendency of a model to consistently over-predict or under-predict actual demand, measured as the mean error. While bias metrics only capture the first moment of the error distribution, CRPS penalizes both systematic location errors (bias) and dispersion errors (miscalibrated spread), providing a more complete diagnostic of probabilistic forecast quality.

An autoregressive recurrent neural network developed by Amazon that produces parametric probabilistic forecasts by learning a distribution (e.g., Negative Binomial) over future time steps. CRPS is a natural evaluation metric for DeepAR models because it directly assesses the quality of the learned output distribution against observed values, natively handling the intermittent demand patterns common in retail.