Stratified K-Fold

Unlike standard K-Fold which randomly shuffles and splits data, Stratified K-Fold performs the split while preserving the percentage of samples for each class. The algorithm iterates through each class independently, assigning examples to folds in equal proportions. This guarantees that if the full dataset has a 0.1% fraud rate, every single training and validation fold will also maintain approximately a 0.1% fraud rate, ensuring the model is always evaluated on a representative sample of the rare event.

In financial fraud detection, the minority class often represents less than 1% of transactions. With standard K-Fold, random chance can create a validation fold with zero fraud examples. This renders metrics like recall or precision undefined or zero for that fold, causing extreme variance in evaluation results. Stratified K-Fold eliminates this evaluation lottery, ensuring every fold is a valid mini-representation of the real-world distribution, leading to stable, reliable performance estimates.

The StratifiedKFold class in scikit-learn provides a direct implementation. Key parameters include:

• n_splits: Number of folds (common values are 5 or 10)
• shuffle: Boolean to shuffle data before splitting (should be True to avoid ordering bias)
• random_state: Seed for reproducibility

The split() method requires both the feature matrix X and the target vector y to calculate the stratified indices.

Standard Stratified K-Fold assumes independent and identically distributed (i.i.d.) data. In fraud detection, this assumption often fails. A single fraudster may generate multiple transactions. If those transactions are split across training and validation folds, data leakage occurs. For such cases, combine stratification with group-based splitting (e.g., StratifiedGroupKFold) to ensure all transactions from a single entity remain together in the same fold, preserving both class balance and group integrity.

Standard stratification handles a single discrete target column. For complex fraud scenarios requiring multi-label classification (e.g., simultaneously predicting fraud type and channel), iterative stratification is required. Algorithms like the iterative stratification method attempt to balance the distribution of multiple labels simultaneously, though perfect balance is often combinatorially impossible. Libraries such as scikit-multilearn provide implementations for these advanced use cases.

Stratified K-Fold fails when a class has fewer examples than the number of folds (e.g., 3 fraud cases with 5-fold CV). In this extreme rare-class scenario, no stratification algorithm can place at least one example in every fold. The solution is to reduce the number of folds or use Repeated Stratified K-Fold, which repeats the splitting process with different randomizations to provide a more robust performance estimate despite the scarcity of data.

A data cleaning method that identifies pairs of minimally distanced nearest neighbors of opposite classes. The majority class instance in each pair is removed to clarify the decision boundary.

• Purpose: Reduces class overlap and noise, making the separation between fraudulent and legitimate transactions more distinct.
• Application: Frequently paired with SMOTE in the SMOTETomek hybrid pipeline to clean the augmented dataset.
• Effect: Creates a sharper, less ambiguous classification frontier for models to learn.

The area under the Precision-Recall curve, a performance metric focused exclusively on the minority class. It is more informative than ROC AUC for highly imbalanced fraud datasets.

• Why it matters: ROC AUC can be misleadingly high when true negatives dominate. Precision-Recall AUC penalizes models that generate excessive false positives.
• Calculation: Plots precision against recall at every threshold, with a perfect score of 1.0.
• Baseline: The baseline is the proportion of positives, not 0.5, making it a more honest metric for rare event detection.

A learning paradigm that assigns different misclassification costs to different error types. Missing a fraudulent transaction (false negative) is penalized far more heavily than flagging a legitimate one (false positive).

• Implementation: Achieved via class weights, modified loss functions, or threshold moving.
• Focal Loss: A dynamic cost function that down-weights easy examples and focuses training on hard, misclassified cases.
• Synergy: Used in conjunction with stratified sampling to ensure the model's objective function aligns with business risk tolerance.

An ensemble method that integrates SMOTE into the AdaBoost algorithm. At each boosting iteration, new synthetic minority examples are generated before training the next weak learner.

• Mechanism: Shifts the focus of each successive classifier toward the minority class by providing a freshly balanced dataset.
• Advantage: Combines the bias reduction of boosting with the data-level solution of SMOTE.
• Comparison: Unlike RUSBoost, which discards majority data, SMOTEBoost preserves all original information while augmenting the rare class.

The process of post-processing a classifier's output scores so that a predicted probability of 0.8 truly means an 80% empirical chance of fraud. Critical for accurate risk scoring.

• Problem: Modern models like XGBoost or neural networks often produce distorted, overconfident probabilities.
• Methods: Platt scaling fits a logistic regression on the raw scores; isotonic regression fits a non-parametric step function.
• Validation: Reliability diagrams plot predicted probability against observed frequency, with a perfect diagonal indicating ideal calibration.