Conformal Prediction

Unlike Bayesian methods or Gaussian assumptions, conformal prediction makes no assumptions about the underlying data distribution. The coverage guarantee holds for any exchangeable data, making it robust for real-world lead time data that is often heavy-tailed, multimodal, or skewed by outlier events like port strikes. This property is critical in supply chains where normality assumptions routinely fail.

Conformal prediction provides a marginal coverage guarantee that is mathematically proven to hold for any sample size, not just asymptotically. If you specify a 90% confidence level, the true lead time will fall within the prediction interval at least 90% of the time. This is distinct from methods that only achieve nominal coverage as sample sizes approach infinity, giving planners confidence even with limited historical shipment data.

Conformal prediction operates as a wrapper around any pre-trained model—whether it's a Gradient Boosting Machine, Temporal Fusion Transformer, or ARIMA. It does not require modifying the underlying point predictor. The framework uses a held-out calibration set to learn the empirical distribution of prediction errors, then constructs intervals around new point forecasts. This allows teams to add rigorous uncertainty quantification to existing lead time models without retraining.

Standard conformal prediction produces fixed-width intervals, but adaptive conformal methods adjust interval width based on local difficulty. For lead time forecasting, this means wider intervals for volatile lanes during peak season and tighter intervals for stable, high-volume routes. Techniques include:

• Locally adaptive conformal inference using normalized nonconformity scores
• Conformalized quantile regression that wraps quantile outputs to achieve valid coverage
• Split conformal prediction with conditional variance estimation

The core mechanism of conformal prediction is the nonconformity score—a measure of how unusual a potential label is given a prediction. Common scores for regression include:

• Absolute residual: |y - ŷ|
• Normalized residual: |y - ŷ| / σ̂(x), where σ̂(x) estimates local variability
• Quantile-based score: max(q̂_low - y, y - q̂_high) The choice of nonconformity score directly impacts interval adaptivity and efficiency.

Conformal prediction requires a strict data split into training, calibration, and test sets. The calibration set—never seen during model training—is used to compute nonconformity scores and determine the threshold for interval construction. This separation prevents overfitting and ensures the coverage guarantee remains valid. For time-series lead time data, a temporal split (training on earlier periods, calibrating on later periods) preserves exchangeability assumptions.

A range of values within which a future observation is expected to fall with a specified confidence level. Unlike point forecasts, prediction intervals quantify the uncertainty around each prediction. Conformal prediction generates these intervals with distribution-free validity guarantees, ensuring that a 90% interval contains the true value at least 90% of the time, regardless of the underlying data distribution.

A statistical technique that estimates the conditional median or other quantiles of a response variable. For lead time forecasting, quantile regression can directly model the 5th and 95th percentiles to construct asymmetric prediction intervals. Conformal prediction extends this by calibrating raw quantile estimates to achieve exact coverage, correcting for any systematic over- or under-confidence in the base model.

The property that a model's predicted probabilities or confidence scores align with observed empirical frequencies. A well-calibrated forecaster issuing 80% prediction intervals should contain the true value exactly 80% of the time. Conformal prediction provides a post-hoc calibration layer that transforms any black-box model's heuristic uncertainty estimates into rigorously calibrated intervals without retraining.

A core assumption of conformal prediction requiring that the order of data points does not matter—the joint distribution of observations is invariant under permutation. This is weaker than the standard i.i.d. assumption. In supply chain contexts, exchangeability holds when historical lead time data and future observations are drawn from the same stable process, though concept drift can violate this condition.

A scoring function that quantifies how unusual or atypical a new data point is relative to a calibration set. Common measures include:

• Absolute residual: |y - ŷ| for regression
• Normalized residual: |y - ŷ| / σ(x) to account for heteroscedasticity
• Quantile-based score: max(q_low - y, y - q_high) The choice of nonconformity measure directly impacts the adaptivity and efficiency of the resulting prediction intervals.

A computationally efficient variant that splits the training data into a proper training set and a calibration set. The model is trained once on the proper training set, and nonconformity scores are computed on the held-out calibration set. This avoids the prohibitive cost of full conformal prediction, which requires retraining the model for every candidate label, making inductive conformal prediction practical for deep learning models in production.