Conformalized Gaussian Processes

A standard GP's credible intervals are only valid if the chosen kernel perfectly captures the true data-generating process—a rare occurrence in practice. CGP uses a held-out calibration set to compute nonconformity scores based on the GP's predictive residuals. By applying a conformal correction, the method widens or narrows the GP's intervals to achieve the target coverage, regardless of whether the kernel is correctly specified.

• Problem: Mismatched smoothness or periodicity assumptions lead to overconfident or underconfident intervals.
• Solution: The conformal layer provides a finite-sample distribution-free guarantee, making the final prediction sets robust to model error.

The core statistical promise of CGP is the marginal coverage guarantee. For a user-specified confidence level (e.g., 95%), the probability that the true test label falls within the conformalized prediction set is at least that level. This guarantee holds exactly in finite samples, not just asymptotically, under the sole assumption of exchangeability between the calibration and test data.

• Mechanism: The nonconformity score quantile from the calibration set directly determines the interval width.
• Result: A rigorous, verifiable uncertainty estimate that auditors and regulators can trust.

A naive conformal correction using a constant adjustment factor can destroy the GP's key advantage: heteroscedastic uncertainty (wider intervals in high-noise regions, tighter intervals in low-noise regions). CGP employs normalized nonconformity measures that scale the residual by the GP's local predictive standard deviation. This ensures the conformal correction respects the GP's learned input-dependent noise structure.

• Normalized Score: s(x,y) = |y - μ(x)| / σ(x)
• Benefit: The final prediction sets remain adaptive, expanding only where the data is genuinely noisy, not uniformly.

CGP typically uses the split conformal prediction framework to avoid the computational cost of retraining the GP. The available labeled data is partitioned into three disjoint sets:

• Proper Training Set: Used to optimize the GP's kernel hyperparameters and compute the posterior mean μ(x) and variance σ²(x).
• Calibration Set: Used exclusively to compute the empirical quantile of the normalized nonconformity scores.
• Test Set: New points where the conformalized prediction intervals are evaluated.

This separation prevents overfitting and guarantees the exchangeability assumption holds for the calibration and test points.

While Conformalized Bayesian Neural Networks apply a similar calibration wrapper to BNN posterior predictive distributions, CGP offers distinct advantages in small-data regimes. The GP's analytical marginal likelihood provides a principled way to optimize kernel parameters without the complex approximate inference required for BNNs. The conformal step then corrects for any remaining model mismatch.

• GP Advantage: Closed-form inference and natural interpolation properties.
• Conformal Benefit: Both methods gain finite-sample guarantees, but CGP is often more computationally efficient for low-to-medium dimensional regression tasks.

CGP inherits the fundamental limitation of all marginal conformal methods: it guarantees coverage on average over the test distribution, not for every specific input x. Achieving conditional coverage P(Y_test ∈ C(X_test) | X_test = x) ≥ 1-α is impossible without strong distributional assumptions. In practice, CGP may undercover in certain regions of the input space while overcovering in others.

• Mitigation: Use Mondrian conformal prediction to achieve group-conditional coverage for pre-defined feature strata.
• Trade-off: Tighter conditional guarantees generally require larger prediction sets.