Energy Disaggregation Accuracy Metrics

The harmonic mean of precision and recall, providing a balanced measure of an algorithm's ability to correctly identify when an appliance is active while minimizing false positives and false negatives.

• Precision: Of all the times the model predicted the appliance was ON, what fraction was actually ON?
• Recall: Of all the times the appliance was actually ON, what fraction did the model correctly detect?
• F1 Formula: 2 * (Precision * Recall) / (Precision + Recall)
• Use Case: Best for evaluating state detection accuracy, especially on imbalanced datasets where appliances are OFF most of the time.

The average absolute difference between the predicted power consumption and the ground truth power consumption for a specific appliance, measured in watts.

• Interpretation: An MAE of 50W means the model's power estimate is, on average, off by 50 watts at each time step.
• Sensitivity: Penalizes large errors linearly, making it robust to outliers compared to RMSE.
• Granularity: Typically calculated at the same sampling rate as the input data (e.g., 1-second or 1-minute intervals).
• Limitation: Does not distinguish between always-predicting-zero and genuine state confusion.

The fraction of total aggregate energy consumption that is correctly attributed to the correct appliance over a defined period, typically expressed as a percentage.

• Formula: 1 - (Sum of Absolute Attribution Errors) / (2 * Total Aggregate Energy)
• Interpretation: A TECA of 85% means 85% of the total energy consumed was assigned to the right device; the remaining 15% was either unassigned or misassigned.
• Strength: Provides a single, intuitive number for overall disaggregation quality across all appliances.
• Weakness: Can mask poor performance on low-power devices if high-power appliances dominate the aggregate signal.

The ratio of the mean squared error of the disaggregated signal to the mean squared error of a naive baseline that always predicts the mean power of the appliance.

• Formula: NDE = Sum((y_true - y_pred)^2) / Sum((y_true - y_mean)^2)
• Interpretation: An NDE of 0 indicates perfect disaggregation; an NDE of 1 means the model performs no better than always guessing the average load.
• Benchmarking: The standard metric used in academic literature to compare models against the REDD and UK-DALE datasets.
• Advantage: Normalizes for appliance power magnitude, enabling fair comparison between high-power and low-power devices.

A balanced classification metric that accounts for all four confusion matrix categories—true positives, true negatives, false positives, and false negatives—producing a score between -1 and +1.

• +1: Perfect prediction.
• 0: No better than random guessing.
• -1: Total disagreement between prediction and observation.
• Why NILM Uses It: Unlike F1-score, MCC only produces a high score if the model performs well on both the majority class (OFF) and the minority class (ON), making it the most reliable single metric for imbalanced appliance state data.

A set of metrics specifically evaluating the algorithm's ability to correctly identify state transitions—the exact moment an appliance turns ON or OFF—rather than steady-state consumption.

• True Positive Event: Detected within a tolerance window (e.g., ±5 seconds) of the actual transition.
• False Positive Event: A transition detected where none occurred.
• False Negative Event: A real transition that the model missed entirely.
• Metrics: Precision, Recall, and F1 calculated on event instances, not time samples.
• Critical For: Real-time feedback applications where users expect immediate notification of appliance state changes.

The harmonic mean of precision and recall, providing a balanced measure of a model's accuracy in identifying appliance states.

• Precision: Of all the times the model predicted an appliance was ON, how often was it actually ON?
• Recall: Of all the times the appliance was actually ON, how often did the model correctly detect it?
• F1 Formula: 2 * (Precision * Recall) / (Precision + Recall)

An F1-score of 0.90+ is considered excellent for NILM, though performance often degrades on low-power or multi-state appliances.

The average absolute difference between the predicted power consumption and the ground truth power consumption for an appliance, measured in watts.

• Interpretation: Lower MAE indicates better estimation accuracy.
• Formula: MAE = (1/n) * Σ|predicted_i - actual_i|
• Use Case: MAE is preferred over RMSE when you want a linear penalty for errors, making it less sensitive to large outlier spikes.

A typical target for major appliances is an MAE below 50 watts.

The percentage of total aggregate energy that is correctly attributed to the right appliance over a defined period.

• Calculation: TECA = 1 - (Σ|predicted_energy_i - actual_energy_i|) / (2 * total_aggregate_energy)
• Significance: Unlike F1-score, TECA accounts for the magnitude of energy consumption, penalizing errors on high-power appliances more heavily.
• Business Value: Directly translates model accuracy into utility billing impact, making it a favorite metric for energy efficiency program managers.

The ratio of the squared error in predicted appliance power to the variance of the ground truth signal.

• Formula: NDE = Σ(predicted_t - actual_t)² / Σ(actual_t - mean_actual)²
• Interpretation: An NDE of 1.0 means the model performs no better than simply guessing the mean power. An NDE of 0.0 is perfect prediction.
• Advantage: NDE normalizes for appliance variability, allowing fair comparison across different appliance types and households.

A tabular visualization breaking down model predictions into True Positives (TP), False Positives (FP), True Negatives (TN), and False Negatives (FN).

• TP: Appliance ON, predicted ON.
• FP: Appliance OFF, predicted ON (phantom load).
• TN: Appliance OFF, predicted OFF.
• FN: Appliance ON, predicted OFF (missed detection).

This matrix is the foundation for calculating precision, recall, and F1-score, and helps diagnose specific failure modes like over-sensitivity or under-detection.

Metrics specifically evaluating the model's ability to correctly identify state transitions (ON/OFF events) rather than steady-state power levels.

• True Positive Rate (TPR): Percentage of real events correctly detected.
• False Positive Rate (FPR): Percentage of detected events that were not real.
• F1-Event: Harmonic mean of event-level precision and recall.

Event-based metrics are critical for applications like real-time alerting and demand response, where missing a turn-on event is more costly than a small steady-state error.