Bad Data Detection

The primary mechanism for detecting bad data by examining the difference between raw measurements and the estimated state. When a sensor malfunctions or a communication error injects a spurious value, it creates an anomalously large measurement residual that violates expected statistical distributions.

• Normalized Residual Test: Divides each residual by its expected standard deviation; values exceeding a threshold (typically ±3σ) are flagged as suspect
• Chi-Square Test: Evaluates the collective sum of squared residuals against a statistical threshold to detect the presence of bad data in the measurement set
• Largest Normalized Residual: An iterative approach that identifies and removes the single most egregious measurement, then re-runs state estimation until all residuals pass validation

Bad data manifests in distinct patterns that require different detection strategies. Understanding the taxonomy of errors helps engineers configure appropriate filtering logic.

• Single Bad Data: One isolated erroneous measurement, typically caused by a single transducer failure or communication packet corruption; easily identified and removed
• Multiple Non-Interacting Bad Data: Several bad measurements that affect different parts of the network and do not mask each other; standard residual tests remain effective
• Multiple Interacting Bad Data: Errors that are topologically adjacent or conforming, where one bad measurement can make another appear valid; requires hypothesis testing or combinatorial search to disentangle
• Critical Measurements: Points where no redundancy exists; bad data at these locations is undetectable because there is no alternative measurement to reveal the discrepancy

The fundamental prerequisite for bad data detection. Without redundant measurements, there is no statistical basis to distinguish a true state change from a sensor error.

• Local Redundancy: Multiple measurements at or near the same bus, such as a direct voltage reading plus an adjacent power flow that implies voltage
• Topological Redundancy: Measurements distributed across the network such that Kirchhoff's laws provide implicit cross-validation; a bad power injection at one bus creates inconsistent flows elsewhere
• Critical Measurement Identification: Pre-processing algorithms analyze the measurement Jacobian matrix to identify buses or branches where redundancy is zero, alerting operators to blind spots in observability

Heuristic checks applied before state estimation to catch obviously impossible values, reducing computational load on the main estimator.

• Range Checks: Reject measurements that fall outside physically plausible limits, such as a voltage magnitude of 0.0 pu or 2.5 pu on a nominal 1.0 pu system
• Rate-of-Change Limits: Flag values that change faster than the physical system can respond, indicating a stuck transducer or intermittent communication fault
• Consistency Checks: Compare measurements against neighboring values using simplified circuit relationships; a power flow reading that contradicts both adjacent voltage readings is suspect
• Timestamp Validation: Discard measurements with stale or misaligned GPS timestamps that would corrupt time-synchronized state estimation

An alternative to explicit bad data removal that uses robust statistical estimators inherently resistant to outliers. These methods automatically down-weight anomalous measurements during the estimation process rather than requiring a separate detection-and-removal step.

• Least Absolute Value (LAV): Minimizes the sum of absolute residuals instead of squared residuals, reducing the influence of outliers on the final estimate
• Huber M-Estimator: Applies quadratic weighting to small residuals and linear weighting to large residuals, providing a smooth transition between normal and outlier treatment
• Least Median of Squares (LMS): Minimizes the median of squared residuals, achieving a theoretical breakdown point of 50%—meaning nearly half the measurements can be bad before the estimator fails
• Schweppe-Type Huber Estimator: A variant that applies Huber weighting in a leverage-aware manner, preventing high-leverage measurements from unduly influencing results

A critical distinction in grid diagnostics. Topology errors occur when the assumed breaker status is incorrect, causing the network model itself to be wrong. Measurement errors occur when the model is correct but the sensor value is wrong.

• Normalized Lagrange Multiplier Test: A statistical method that can distinguish between a bad analog measurement and an incorrect breaker status by testing both hypotheses simultaneously
• Generalized State Estimation: An advanced formulation that estimates breaker statuses alongside bus voltages, treating topology as a variable rather than a fixed input
• Suspected Bad Data Zones: When multiple adjacent measurements fail residual tests, the root cause is often a topology error rather than simultaneous sensor failures

The algorithmic process that computes the most likely operational state of a power grid by filtering noisy, redundant, and asynchronous sensor measurements against a network model. Bad data detection is an integral sub-function of the state estimator, using residual analysis to identify and reject grossly erroneous measurements before they corrupt the solution. The estimator solves an overdetermined system of equations, minimizing the weighted sum of squared residuals between measured and calculated values.

A topological assessment that determines whether the available set of measurements is sufficient to uniquely estimate the voltage magnitude and angle at every bus in the network model. Without full observability, bad data detection becomes unreliable because the redundancy required for residual-based identification is absent. The analysis identifies observable islands and critical measurements whose loss would render portions of the grid unobservable.

A steady-state optimization technique that minimally adjusts raw process measurements to satisfy known physical conservation laws, such as Kirchhoff's current and voltage laws. This provides a consistent dataset for model calibration and is closely related to bad data detection—both rely on the principle that gross errors violate physical constraints in statistically improbable ways. The reconciled values represent the best estimate of true process conditions.

The computational integration of data from disparate measurement sources—SCADA, PMUs, and smart meters—to produce a more accurate and reliable estimate of grid state than any single source. Effective sensor fusion depends on robust bad data detection to prevent a single faulty sensor from contaminating the fused estimate. Techniques include weighted averaging, Kalman filtering, and Bayesian inference.

A recursive mathematical algorithm that estimates the dynamic state of a system from a stream of noisy measurements. The extended Kalman filter and unscented Kalman filter variants are widely used for real-time tracking of grid voltage and angle dynamics. The filter's innovation sequence—the difference between predicted and actual measurements—naturally supports bad data detection through statistical tests for outliers in the residual stream.

The rigorous mathematical characterization of confidence bounds around a digital twin's predictions, distinguishing between aleatoric uncertainty from sensor noise and epistemic uncertainty from model gaps. Bad data detection directly addresses aleatoric uncertainty by identifying measurements whose error distributions fall outside expected bounds. Together, these disciplines ensure that grid operators understand not just the estimated state, but how much to trust it.