Conformal e-values

An e-value is a nonnegative random variable E satisfying E[E] ≤ 1 under the null hypothesis. Unlike p-values, e-values remain valid under optional stopping and optional continuation, meaning you can peek at results, stop early, or continue collecting data without invalidating inference. This property makes them ideal for sequential testing and online conformal prediction scenarios where data arrives in streams.

Traditional conformal p-values require a fixed sample size determined in advance. Conformal e-values enable anytime-valid guarantees: at any stopping time τ, the product of sequential e-values forms an e-process that maintains validity. Key properties:

• Ville's inequality replaces Markov's inequality for sequential bounds
• No correction for multiple looks at data required
• Valid under arbitrary stopping rules, including data-dependent ones

Multiple independent e-values can be combined using e-merging functions to produce a single valid e-value. Common merging strategies include:

• Arithmetic mean: Simple averaging preserves validity
• Product merging: Multiplying independent e-values yields a valid combined e-value
• Calibrated merging: Weighted combinations for heterogeneous evidence sources This enables meta-analysis and federated conformal inference across distributed data silos.

Conformal e-values produce e-prediction sets with a different guarantee than traditional conformal prediction. Instead of marginal coverage at level 1-α, e-prediction sets control the expected false coverage rate: the expected proportion of incorrect labels included is bounded. This provides:

• Sharper sets in many practical scenarios
• Natural integration with cost-sensitive decision-making
• Direct compatibility with sequential decision processes

E-values and p-values are connected through calibration:

• Any e-value E can be converted to a p-value via p = min(1, 1/E)
• The reverse requires a calibrator function, which always introduces some conservatism
• E-values are more conservative than p-values for fixed-sample testing but gain power through sequential aggregation
• The GRO (Growth Rate Optimal) e-value maximizes expected logarithmic growth under the alternative

In online conformal inference, e-values enable continuous monitoring with valid error control:

• Sequential e-values are computed for each new observation
• The running product forms a test martingale under the null
• Rejection occurs when the product exceeds 1/α
• Applications include real-time anomaly detection, A/B testing with peeking, and continual model monitoring without pre-specified sample sizes