Ensemble Learning

A parallel ensemble method that trains multiple instances of the same base learner on different random subsets of the training data, then aggregates predictions through majority voting or averaging.

Mechanism:

• Creates diverse training sets via bootstrap sampling (random sampling with replacement)
• Each model trains independently, enabling full parallelization
• Reduces variance without increasing bias—ideal for high-variance learners like decision trees

Transformer Application: Random Forest, the canonical bagging algorithm, excels at Failure Mode Classification using DGA ratios. By averaging hundreds of decision trees, it smooths out the noise inherent in online DGA monitor readings and provides robust fault type predictions even with missing gas values.

A sequential ensemble technique where each new model is trained to correct the errors made by its predecessors, converting a set of weak learners into a single strong predictor.

Mechanism:

• Models are trained sequentially, with each iteration assigning higher weight to previously misclassified samples
• Focuses on reducing bias by iteratively refining decision boundaries
• Gradient Boosting and XGBoost use gradient descent to minimize a differentiable loss function

Transformer Application: XGBoost is the dominant algorithm for Remaining Useful Life (RUL) prediction. It captures complex, non-linear relationships between Hot-Spot Temperature trajectories, Moisture Content, and Degree of Polymerization degradation rates, outperforming traditional Weibull Distribution models in accuracy.

A meta-learning architecture that combines predictions from multiple diverse base models using a higher-level meta-learner, which learns the optimal way to blend their outputs.

Mechanism:

• Level-0: Train heterogeneous base models (e.g., Random Forest, SVM, Gradient Boosting) on the same dataset
• Level-1: A meta-model (often logistic regression or a shallow neural network) learns to weight each base model's predictions based on their reliability in different regions of the feature space
• Exploits the complementary strengths of fundamentally different algorithms

Transformer Application: Stacking integrates a Physics-Informed Neural Network (PINN) with a statistical XGBoost model. The PINN constrains predictions to thermodynamic laws, while XGBoost captures empirical patterns from historical failure data. The meta-learner weights the PINN more heavily when operating conditions deviate from historical norms, ensuring physical plausibility.

The simplest ensemble architecture that aggregates predictions from multiple heterogeneous models through hard voting (majority rule) or soft voting (averaging predicted probabilities).

Mechanism:

• Hard Voting: Each model casts one vote for a class label; the class with the most votes wins
• Soft Voting: Each model outputs a probability for each class; probabilities are averaged, and the class with the highest mean probability is selected
• Soft voting is generally superior when models are well-calibrated, as it accounts for prediction confidence

Transformer Application: A soft voting ensemble combining an Autoencoder (anomaly detection), a Duval Triangle classifier (DGA interpretation), and a LSTM time-series forecaster provides multi-modal fault detection. The Autoencoder flags operational anomalies, the Duval classifier identifies specific fault types, and the LSTM predicts impending gas level excursions—together providing both detection and diagnostic context.

A staged architecture where models are arranged sequentially, with each stage filtering out easy cases and passing only ambiguous or high-stakes instances to more computationally expensive downstream models.

Mechanism:

• Stage 1: A lightweight, low-latency model (e.g., a simple threshold-based Health Index) screens all incoming data
• Stage 2: Only instances flagged as borderline or anomalous are passed to a more sophisticated ensemble (e.g., a stacked model)
• Optimizes computational resource allocation—critical for Edge AI deployments on substation gateways

Transformer Application: Deployed on an Edge AI device at a substation, a cascading ensemble first applies a fast statistical anomaly detector to online DGA monitor streams. Only when gas ratios exceed IEC 60599 normal limits does it invoke a full XGBoost Failure Mode Classification model, conserving limited computational resources while maintaining high diagnostic fidelity for critical events.

A variant of stacking that uses a holdout validation set to train the meta-learner, rather than cross-validation, simplifying the training pipeline while reducing the risk of data leakage.

Mechanism:

• Split training data into a base-training set and a holdout set
• Train base models on the base-training set
• Generate predictions on the holdout set to create the meta-model's training features
• The meta-model never sees the data used to train the base models, preventing overfitting

Transformer Application: Blending is preferred when combining models trained on disparate data sources—for example, merging a model trained on Infrared Thermography hotspot data with one trained on Furan Analysis results. The holdout set ensures the meta-learner generalizes to the combined feature space without memorizing correlations specific to the training split.

A bagging-based ensemble of decision trees trained on random subsets of DGA data. Each tree votes on a fault class, and the majority prediction wins. Random Forest excels at handling the high-dimensional, noisy gas concentration data typical of online DGA monitors.

• Reduces overfitting compared to single decision trees
• Provides built-in feature importance rankings for gas ratios
• Robust to missing sensor readings and outliers

A sequential boosting algorithm that builds trees iteratively, with each new tree correcting the residual errors of the previous ensemble. XGBoost dominates transformer fault classification benchmarks due to its handling of class imbalance—critical when arcing faults are rare compared to thermal faults.

• Built-in regularization prevents overfitting on small DGA datasets
• Native handling of missing acetylene or hydrogen readings
• Often outperforms neural networks on tabular sensor data

A meta-learning ensemble where predictions from diverse base models—such as a neural network, SVM, and Random Forest—are fed into a final meta-learner (often logistic regression) that learns how to optimally weight each model's output. Stacking is particularly effective when combining models trained on different diagnostic inputs.

• Base models can include both DGA-based and electrical test-based classifiers
• Meta-learner learns which model to trust for specific fault signatures
• Reduces the risk of relying on a single algorithmic approach

A variance-reduction technique that trains multiple instances of the same algorithm on different bootstrap samples of the transformer condition dataset. By averaging predictions, bagging stabilizes high-variance models that would otherwise overfit to the specific gas patterns of a single transformer.

• Effective when training data is limited to a small fleet
• Each model sees a different random subset of historical DGA records
• Reduces sensitivity to outlier gas spikes from sensor malfunctions

A family of sequential ensemble methods that convert weak learners into a strong predictor by focusing each iteration on the misclassified fault examples. In transformer diagnostics, boosting algorithms like AdaBoost and LightGBM progressively learn to distinguish subtle differences between thermal faults and partial discharge signatures.

• Weights misclassified failure modes more heavily in subsequent rounds
• Effective at learning complex decision boundaries between fault types
• Requires careful tuning to avoid amplifying noise in DGA data

A simple yet powerful ensemble that aggregates predictions from heterogeneous models using either hard voting (majority rule) or soft voting (averaged probabilities). In transformer fault classification, a voting ensemble might combine a physics-informed model with a data-driven XGBoost model to balance domain knowledge and pattern recognition.

• Hard voting: each model gets one vote on fault type
• Soft voting: averages predicted probabilities for each fault class
• Provides a straightforward baseline before deploying complex ensembles