Extreme Value Theory (EVT)

The foundational theorem of EVT, proving that the distribution of block maxima converges to one of three limiting forms: the Gumbel, Fréchet, or Weibull distributions. This is analogous to the Central Limit Theorem but for extremes rather than means.

• Gumbel: Light-tailed distributions (e.g., Gaussian)
• Fréchet: Heavy-tailed distributions (e.g., Cauchy)
• Weibull: Distributions with a finite upper bound

In open set recognition, this theorem justifies using the Weibull family to model the distance between a sample and its class centroid.

An alternative to block maxima that models all observations exceeding a sufficiently high threshold u. The Balkema-de Haan-Pickands theorem states that these exceedances follow a Generalized Pareto Distribution (GPD).

• More data-efficient than block maxima for continuous monitoring
• Threshold selection involves a bias-variance trade-off: too low violates asymptotic theory, too high reduces sample size
• Used in anomaly detection to dynamically set rejection thresholds based on tail behavior of reconstruction errors

A technique that fits a Weibull distribution to the distances between correctly classified training samples and their class mean activation vectors (MAVs). The fitted tail parameters are used to recalibrate the SoftMax output.

• Computes the probability that a sample belongs to the tail of any known class
• Rejects samples where the cumulative distribution function indicates extreme outlier status
• Directly addresses open space risk by bounding the region classified as known
• Enables a principled rejection mechanism without requiring unknown class samples during training

A unified parameterization combining the Gumbel, Fréchet, and Weibull families into a single distribution governed by a shape parameter ξ (xi).

• ξ = 0: Gumbel domain (light tail)
• ξ > 0: Fréchet domain (heavy tail, infinite moments)
• ξ < 0: Weibull domain (finite endpoint)

In ML, the shape parameter provides diagnostic information about the nature of extreme events in the feature space, guiding whether a model should use bounded or unbounded rejection regions.

A core application of EVT that estimates the quantile expected to be exceeded once every T time periods (the T-period return level). In open set recognition, this translates to setting a threshold such that only a specified fraction of known samples are falsely rejected.

• Return level plot: Diagnostic tool for validating EVT model fit
• Confidence intervals widen dramatically for extrapolation beyond observed data
• Used to calibrate the trade-off between false rejection rate and open space risk
• Provides a principled alternative to arbitrary percentile-based thresholds

The tail index α (alpha) is the reciprocal of the GEV shape parameter and quantifies the heaviness of a distribution's tail. A smaller α indicates a heavier tail with more frequent extreme events.

• α ≤ 2: Infinite variance (e.g., stable distributions)
• α ≤ 1: Infinite mean
• In feature embeddings, heavy-tailed behavior indicates that extreme distances from class centroids are more probable than a Gaussian assumption would predict
• The Hill estimator provides a simple method for estimating α from the k largest order statistics

The canonical algorithm that replaces the standard SoftMax layer with an EVT-calibrated rejection mechanism. OpenMax fits a Weibull distribution to the distance between each training sample and its class mean vector. At inference, it recalibrates activation scores using the Weibull CDF, redistributing probability mass to a pseudo-class representing the unknown. This directly addresses open space risk by estimating the likelihood that a sample belongs to any known class.

The task of identifying inputs that differ fundamentally from the training distribution. Unlike closed-set misclassification, OOD samples require a rejection decision. EVT provides the theoretical basis for setting rejection thresholds by modeling the tails of the in-distribution activation or feature space. Key approaches include:

• Energy-based models that assign high energy scores to OOD inputs
• Mahalanobis distance in feature space with EVT-calibrated thresholds
• Monte Carlo Dropout for Bayesian uncertainty estimation

The process of aligning a model's predicted probability with its empirical accuracy. A well-calibrated model that outputs 0.9 confidence should be correct 90% of the time. Temperature scaling is the most common post-hoc method, dividing logits by a learned scalar. EVT enhances calibration for open set scenarios by modeling the extreme value distribution of maximum activation scores, enabling statistically rigorous rejection thresholds rather than arbitrary confidence cutoffs.

A low-dimensional vector representation where semantic similarity maps to geometric proximity. The quality of the embedding space directly determines EVT's effectiveness—compact, well-separated class clusters produce reliable tail models. Techniques that enforce this structure include:

• Angular margin losses (ArcFace, CosFace) that maximize inter-class separation
• Contrastive learning that pulls similar samples together and pushes dissimilar apart
• Prototypical networks that learn a metric space where classes cluster around prototypes

The reducible uncertainty arising from a lack of knowledge or data. It is high for inputs far from the training distribution—precisely the regime where EVT operates. Evidential deep learning places a Dirichlet distribution over class probabilities to jointly model evidence, belief mass, and uncertainty. EVT complements this by providing the asymptotic theory for the tails of the uncertainty distribution, enabling principled thresholds for when epistemic uncertainty signals an unknown emitter.

A distribution-free framework that produces prediction sets with guaranteed marginal coverage probability. Unlike EVT's parametric approach, conformal prediction makes no distributional assumptions. It uses a calibration set to determine nonconformity scores, then outputs sets containing the true class with a user-specified probability (e.g., 95%). For open set recognition, conformal prediction provides a rigorous statistical basis for rejecting unknown classes when the prediction set is empty or exceeds a size threshold.