SMOTE

SMOTE creates synthetic samples by interpolating between a selected minority class instance and one of its k-nearest minority class neighbors. For each new sample, it:

• Selects a random neighbor from the k-nearest neighbors
• Calculates the vector difference between the feature vectors
• Multiplies this difference by a random number between 0 and 1
• Adds the result to the original instance

This produces a new point along the line segment connecting the two original points, expanding the minority class decision region rather than reinforcing exact copies.

Unlike random oversampling, which duplicates existing minority examples and causes models to memorize specific instances, SMOTE generates previously unseen synthetic samples. This forces the classifier to learn broader, more generalizable regions for the minority class. By populating the feature space between real examples, SMOTE reduces the variance of the decision boundary and helps prevent the model from treating the minority class as a collection of isolated points. This is critical in fraud detection where novel fraud patterns must be recognized.

SMOTE operates purely in the feature space defined by the dataset's continuous variables. It does not consider the target variable during generation, meaning it can create ambiguous samples near the decision boundary. Key operational characteristics:

• Requires all features to be numerical; categorical features must be encoded first
• The choice of k (number of neighbors) controls the diversity of generated samples
• Standard implementation uses k=5 as a default
• Performs best when combined with cleaning methods like Tomek Links or Edited Nearest Neighbors to remove overlapping synthetic points

Several SMOTE variants address specific weaknesses of the original algorithm:

• Borderline-SMOTE: Only oversamples minority instances near the decision boundary, ignoring interior points that are easy to classify
• SVM-SMOTE: Uses a Support Vector Machine to identify the optimal boundary and generates samples along it
• ADASYN: Adaptively generates more samples for minority instances that are harder to learn, based on the density of surrounding majority class neighbors
• SMOTE-NC: Extends SMOTE to handle datasets with both numerical and categorical features by using a modified distance metric

SMOTE is frequently combined with under-sampling techniques to create cleaner class separation:

• SMOTETomek: Applies SMOTE first, then removes overlapping majority class instances using Tomek Links. This clarifies the boundary by eliminating majority points that invade the minority region
• SMOTEENN: Generates synthetic samples with SMOTE, then applies Edited Nearest Neighbors to remove any example (from either class) whose predicted label conflicts with its nearest neighbors

These hybrid approaches address SMOTE's tendency to create noise and ambiguous boundary samples.

SMOTE's computational complexity scales with the size of the minority class and the chosen k parameter. Key performance factors:

• O(n²) worst-case complexity for nearest neighbor search without optimized indexing
• Memory usage increases linearly with the number of synthetic samples generated
• For large-scale fraud detection with millions of transactions, consider approximate nearest neighbor methods or batch processing
• The sampling ratio (e.g., 200%, 300%) directly controls the final dataset size and training time
• Modern implementations in imbalanced-learn use optimized data structures for efficiency