Conformalized Quantile Regression

CQR inherits the core guarantee of conformal prediction: finite-sample marginal coverage. Unlike raw quantile regression, which relies on asymptotic assumptions and can be miscalibrated in practice, CQR provides a rigorous, nonparametric guarantee.

• No error distribution assumed: Works for heteroskedastic, heavy-tailed, or multimodal data.
• Guarantee: For a target coverage of 90%, the true value falls within the CQR interval at least 90% of the time, averaged over the calibration and test data.
• Key mechanism: The conformal step corrects the raw quantile estimates (e.g., 5th and 95th percentiles) by adding a calibration-derived buffer.

CQR produces heteroskedastic prediction intervals that naturally widen in regions of high uncertainty and narrow where the model is confident. This adaptivity is inherited from the underlying quantile regression model.

• Captures local uncertainty: Unlike methods that produce constant-width intervals, CQR reflects varying data density and noise levels across the input space.
• Example: In financial forecasting, CQR intervals automatically widen during volatile market regimes and tighten during stable periods.
• Comparison: Standard split conformal prediction with absolute residuals produces intervals of fixed width, losing this local adaptivity.

CQR operates in a distinct train-calibrate-predict pipeline that separates model fitting from uncertainty calibration.

• Stage 1 (Training): Fit a quantile regression model on the proper training set to estimate the lower and upper conditional quantiles (e.g., 0.05 and 0.95).
• Stage 2 (Calibration): Compute nonconformity scores on a held-out calibration set. The score is typically the maximum of how far each observation falls below the lower quantile or above the upper quantile.
• Stage 3 (Prediction): For a new point, compute the raw quantile interval and expand it by the empirical quantile of the calibration nonconformity scores.

The nonconformity measure is the engine of CQR's correction. The standard choice is:

S(x, y) = max{ q̂_α(x) - y, y - q̂_{1-α}(x) }

• Interpretation: Measures how far the true label y falls outside the raw prediction interval [q̂_α(x), q̂_{1-α}(x)].
• Symmetric penalty: Penalizes violations of both the lower and upper bound equally.
• Asymmetric variants: Can be modified to prioritize coverage on one tail (e.g., avoiding underestimation in safety-critical systems) by applying different weights to lower and upper violations.

Standard CQR assumes exchangeability between calibration and test data. When this assumption is violated—such as under covariate shift—Weighted CQR extends the framework.

• Weighted calibration: Applies importance weights to calibration nonconformity scores, where weights estimate the density ratio p_test(x) / p_cal(x).
• Guarantee: Maintains valid coverage under known or estimated covariate shift.
• Application: Deploying a CQR model in a new geographic region where the input distribution differs from the training environment.

CQR is model-agnostic and integrates seamlessly with neural networks by modifying the loss function to simultaneously estimate multiple quantiles.

• Pinball loss: The standard quantile regression loss L_τ(y, q̂_τ) = max(τ(y - q̂_τ), (τ-1)(y - q̂_τ)) is used to train a single network with multiple output heads, one per target quantile.
• Conformalized Deep Ensembles: Combining CQR with deep ensembles further improves adaptivity by leveraging ensemble variance before applying the conformal correction.
• End-to-end pipeline: Train a quantile neural network, calibrate on a held-out set, and deploy with guaranteed coverage—all within a standard deep learning framework.

The scoring function that defines what constitutes an unusual prediction. In CQR, the nonconformity measure is typically the signed error between the observed value and the predicted quantile bounds:

• max(ŷ_lower - y, y - ŷ_upper) This captures how far the true label falls outside the estimated interval. The choice of nonconformity measure directly shapes the size and adaptivity of the final prediction sets. Alternative measures can prioritize symmetric expansion or asymmetric coverage.

The computational strategy that makes CQR practical. The data is partitioned into three disjoint sets:

• Training set: Fit the base quantile regression model
• Calibration set: Compute nonconformity scores and determine the correction factor
• Test set: Apply the calibrated intervals to new points This split avoids the prohibitive cost of retraining the model for each calibration point, enabling single-pass calibration while preserving the finite-sample coverage guarantee.

The statistical promise that CQR delivers: P(Y_test ∈ C(X_test)) ≥ 1 - α This holds on average over all test points, not conditionally on any specific feature value. The guarantee is distribution-free and finite-sample—it requires only that the calibration and test data are exchangeable. For a 90% target coverage, CQR ensures the true value falls within the prediction interval at least 90% of the time across the test distribution.

The base estimator that CQR wraps with conformal calibration. Standard quantile regression estimates the conditional quantile function Q_τ(x) by minimizing the pinball loss:

• ρ_τ(u) = u(τ - I(u < 0)) Fitting models at τ = α/2 and τ = 1 - α/2 produces initial prediction intervals. However, these intervals lack finite-sample guarantees—they rely on asymptotic assumptions and correct model specification. CQR corrects this by calibrating the raw quantile estimates against a held-out set.

The stronger, more elusive objective beyond marginal guarantees. Conditional coverage requires P(Y ∈ C(x) | X = x) ≥ 1 - α for every specific feature vector x. CQR with marginal calibration cannot achieve this exactly without additional assumptions—a fundamental limitation proven by Vovk. However, CQR's adaptive interval widths, inherited from quantile regression, provide approximate conditional coverage that outperforms fixed-width conformal methods in heteroscedastic settings.