Conformal Prediction

Unlike Bayesian methods that require assumptions about data distributions, conformal prediction provides finite-sample marginal coverage guarantees without any distributional assumptions. For a user-specified significance level α (e.g., 0.1), the prediction set will contain the true value with probability at least 1-α. This holds for any underlying data distribution, making it robust for ADMET prediction where molecular property distributions are often non-Gaussian and heavy-tailed.

Conformal prediction operates as a post-hoc calibration layer that wraps around any pre-trained model—whether a graph neural network, random forest, or transformer—without modifying its internal architecture. This decoupling means you can apply rigorous uncertainty quantification to existing QSAR models, ChemBERTa embeddings, or DeepChem pipelines without retraining. The only requirement is a held-out calibration set of labeled examples not used during training.

Rather than outputting a single scalar value (e.g., 'LogP = 3.2'), conformal prediction produces a prediction interval or set that contains the true value with a specified confidence level. For regression tasks like solubility prediction, this yields an interval [2.8, 3.6]; for classification tasks like AMES mutagenicity, it may return a set of possible classes. This is critical in drug discovery, where knowing the range of plausible values informs go/no-go decisions more reliably than a point estimate.

The core theoretical requirement for conformal prediction is exchangeability—the assumption that the order of calibration and test data points does not matter. Formally, the joint distribution of the data is invariant under permutation. In practice, this means calibration and test data must be drawn from the same distribution. For molecular property prediction, this requires careful attention to the applicability domain: predictions on compounds far from the calibration set's chemical space may violate exchangeability and degrade coverage guarantees.

The engine of conformal prediction is the nonconformity score—a function that quantifies how unusual a given prediction is relative to the calibration data. Common choices include:

• Absolute residual for regression: s(x,y) = |y - ŷ|
• 1 - softmax probability for classification
• Mahalanobis distance for multivariate outputs In molecular applications, domain-specific nonconformity measures can incorporate molecular similarity or applicability domain distance to produce tighter, more informative prediction sets.

Transductive (full) conformal prediction requires retraining the model for every new test point, which is computationally prohibitive for deep learning. Inductive (split) conformal prediction solves this by splitting the training data once, training the model on the proper training set, and using a separate calibration set to compute nonconformity scores. This single-pass calibration makes it practical for large-scale virtual screening campaigns where millions of compounds must be evaluated with valid confidence intervals.