Random Undersampling

Random Undersampling works by randomly selecting and removing examples from the majority class until the desired class distribution ratio is achieved. The selection process is uniform and unbiased, meaning every majority class instance has an equal probability of being discarded. The minority class remains completely untouched. The result is a smaller, perfectly balanced dataset where both classes have an equal number of instances, forcing the classifier to stop ignoring the rare class.

The most significant drawback is the discarding of potentially useful data. Removed majority class examples may contain critical information about the decision boundary, legitimate transaction patterns, or feature distributions. In extreme imbalances—such as fraud detection with a 0.1% positive rate—undersampling can delete over 99% of the available data, leaving the model with a tiny, non-representative training set that fails to capture the true variance of legitimate behavior.

Random Undersampling dramatically reduces training time and memory requirements by shrinking the dataset size. For large-scale financial transaction systems with millions of records, this can reduce training time from hours to minutes. The technique requires no distance calculations, no synthetic data generation, and no iterative optimization, making it the fastest resampling method available. This efficiency makes it ideal for rapid prototyping and baseline model development.

By reducing the dataset size, Random Undersampling increases model variance. The specific random seed used for sampling can significantly impact which majority class examples survive, leading to unstable model performance across different runs. However, because it eliminates duplicate or near-duplicate majority class examples, it can actually reduce overfitting compared to Random Oversampling, which memorizes exact copies of minority class instances.

To mitigate information loss, Random Undersampling is frequently embedded within ensemble frameworks like EasyEnsemble and RUSBoost. These methods train multiple classifiers on independently undersampled subsets of the majority class, ensuring that every majority class example is used in at least one model. The ensemble aggregates predictions, recovering the information discarded by any single undersampling run while maintaining balanced training sets for each weak learner.

Random Undersampling is most appropriate when:

• The majority class is extremely large and information loss is acceptable
• You need a fast baseline before exploring more sophisticated methods
• The dataset has low noise and clear class separation
• You are using ensemble methods that can recover discarded information

It is generally not recommended as a standalone solution when the dataset is already small or when the majority class has high variance that must be preserved.

A family of informed under-sampling algorithms that select majority class examples based on their average distance to the nearest minority class instances. Unlike random undersampling, NearMiss keeps majority examples that are closest to the decision boundary, preserving information critical for classification.

• NearMiss-1: Keeps majority examples with the smallest average distance to the three closest minority examples
• NearMiss-2: Keeps majority examples with the smallest average distance to the three furthest minority examples
• NearMiss-3: Selects a fixed number of majority examples for each minority example

Synthetic Minority Over-sampling Technique generates new minority class examples by interpolating between existing instances in feature space. Rather than duplicating data, SMOTE creates synthetic samples along line segments connecting a minority example to its k-nearest minority neighbors.

• Addresses overfitting risk inherent in random oversampling
• Expands the minority class decision region
• Often combined with undersampling for optimal results (e.g., SMOTETomek)

A data cleaning method that identifies pairs of minimally distanced nearest neighbors belonging to opposite classes. By removing the majority class instance from each Tomek Link pair, the technique clarifies the decision boundary and reduces class overlap.

• Acts as a targeted undersampling strategy
• Removes ambiguous majority examples near the boundary
• Frequently paired with SMOTE as a post-processing cleanup step

An undersampling technique that removes any majority class example whose predicted class (based on its k-nearest neighbors) differs from its actual class. This eliminates noisy or ambiguous instances that confuse the classifier.

• Default k=3 neighbors for classification
• Can be applied iteratively (Repeated ENN) for more aggressive cleaning
• Often used as a component in hybrid methods like SMOTEENN

A hybrid ensemble algorithm combining Random Under-Sampling (RUS) with the AdaBoost boosting procedure. Each weak learner is trained on a newly undersampled, balanced dataset, allowing the ensemble to recover information lost through individual undersampling runs.

• Trains multiple classifiers on different majority class subsets
• Combines predictions via weighted voting
• Outperforms standalone random undersampling on most benchmarks

The area under the Precision-Recall curve, a performance metric that focuses exclusively on the minority class. Unlike ROC AUC, PR AUC is insensitive to true negatives, making it far more informative for evaluating models on highly imbalanced datasets where the negative class dominates.

• Precision = True Positives / (True Positives + False Positives)
• Recall = True Positives / (True Positives + False Negatives)
• A baseline random classifier achieves AUC = minority class proportion