Analog Ensemble (AnEn)

The core mechanism involves searching a historical dataset of past numerical weather prediction (NWP) forecasts for a set of analogs—past forecast states that are most similar to the current target forecast. Similarity is defined by a multivariate metric across predictor variables like pressure, temperature, and humidity.

• Predictor Selection: Variables are chosen based on their physical correlation with the target predictand (e.g., solar irradiance).
• Temporal Matching: Searches are constrained to a time window around the target valid time to preserve diurnal and seasonal cycles.

Once the top-k analog forecast states are identified, the method retrieves the corresponding historical observations that actually occurred following those analog forecasts. These observations form the predictive distribution for the current target.

• Bias Correction: Inherently corrects for systematic NWP biases because it maps forecast states directly to observed outcomes.
• Non-Parametric: Makes no assumptions about the underlying probability distribution of the predictand.

The set of retrieved observations constitutes an ensemble that directly represents the predictive probability density function (PDF). Quantiles, prediction intervals, and full distributions are derived without needing separate statistical post-processing.

• Quantile Extraction: The 10th, 50th, and 90th percentiles are simply the corresponding percentiles of the analog observation set.
• Uncertainty Quantification: The spread of the analog observations naturally reflects the flow-dependent forecast uncertainty.

AnEn is significantly less computationally intensive than running a full dynamical ensemble prediction system. The heavy lifting is a one-time indexing of the historical archive; real-time prediction reduces to a fast k-nearest neighbor search.

• No Model Retraining: Unlike machine learning models, AnEn does not require iterative training cycles.
• Scalable Search: Modern vector indexing techniques enable sub-second analog retrieval from archives spanning decades.

AnEn is widely applied to predict solar irradiance and wind speed for grid integration. It excels at capturing complex, non-linear relationships between large-scale atmospheric patterns and local renewable generation.

• GHI Prediction: Predictors include 500 hPa geopotential height, total column water vapor, and temperature.
• Wind Power: Predictors include hub-height wind speed and direction from NWP, matched to observed power output.
• Ramp Event Capture: The analog spread often captures the possibility of sudden ramp events better than deterministic models.

The physics-based computational engine that generates the deterministic target forecast used by AnEn to search for analogs. NWP solves discretized equations of atmospheric dynamics—including fluid motion, thermodynamics, and radiative transfer—on a three-dimensional grid. Key characteristics:

• Provides gridded state variables (geopotential height, temperature, wind components) at multiple pressure levels
• Resolution typically ranges from 9km (HRRR) to 28km (GFS)
• Systematic biases in NWP output are implicitly corrected by AnEn's historical matching process
• The quality of the analog search is fundamentally bounded by the fidelity of the driving NWP model

A distinct approach to uncertainty quantification that generates multiple future atmospheric states by perturbing initial conditions, model physics, or boundary conditions. Unlike AnEn—which derives a predictive distribution from a single deterministic forecast by searching history—traditional ensemble forecasting runs the forward model many times. Critical distinctions from AnEn:

• Computationally expensive: requires 10-50 parallel NWP integrations
• Explicitly samples initial condition uncertainty via singular vector or bred vector perturbations
• AnEn achieves similar probabilistic skill at a fraction of the computational cost
• Hybrid approaches often blend AnEn with dynamical ensembles for improved tail-risk characterization

The output format generated by AnEn—a full predictive distribution rather than a single deterministic value. This distribution is constructed from the k historical observations corresponding to the most similar past atmospheric states. Operational value:

• Enables risk-based reserve sizing: grid operators can hold spinning reserve proportional to the 95th percentile of the forecast distribution
• Supports stochastic unit commitment and economic dispatch optimization
• Communicates forecast confidence: a narrow distribution signals high certainty, while a wide spread warns of regime uncertainty
• AnEn naturally produces non-parametric distributions that capture skewness and multimodality without assuming Gaussianity

The strictly proper scoring rule used to evaluate the full predictive distribution produced by AnEn. CRPS measures the integrated squared difference between the forecast cumulative distribution function (CDF) and the empirical observation CDF. Why it matters for AnEn verification:

• Rewards both calibration (statistical consistency between forecasts and observations) and sharpness (concentration of the predictive distribution)
• Reduces to absolute error for deterministic forecasts, enabling direct comparison between probabilistic and point forecasts
• Sensitive to distance: penalizes forecasts that place probability mass far from the observation
• CRPS skill scores relative to climatology or persistence quantify the value added by the analog search procedure

A statistical post-processing technique that corrects systematic biases in raw NWP output by establishing a regression relationship between historical model forecasts and local observations. AnEn can be viewed as a non-parametric, nonlinear generalization of MOS. Comparative framework:

• Classical MOS fits a linear regression (or logistic regression for categorical variables) between NWP predictors and observed predictands
• AnEn replaces the parametric regression with a similarity-weighted empirical distribution
• Both methods require a training archive of paired forecast-observation data
• AnEn naturally handles nonlinear predictor-predictand relationships and non-Gaussian error distributions that challenge traditional MOS formulations

The naive baseline model that assumes current conditions remain unchanged throughout the forecast horizon. For renewable generation, this typically means assuming current power output stays constant. Role in AnEn evaluation:

• Serves as the minimum skill threshold: any operational forecasting system must outperform persistence to justify its complexity
• AnEn typically demonstrates greatest skill improvement over persistence during regime transitions (e.g., frontal passages, sunrise/sunset ramp periods)
• Forecast skill score = 1 - (RMSE_AnEn / RMSE_persistence)
• Persistence skill decays rapidly with horizon; AnEn maintains positive skill scores out to 6-48 hours depending on the variable and region