RuleFit

RuleFit constructs a generalized linear model where the predictors are both the original input features and a set of binary rule indicators extracted from decision tree ensembles. The model applies L1 (Lasso) regularization during training, which drives many coefficients to exactly zero. This produces a sparse final model—typically retaining only 15-50 of the most predictive rules and features—making the entire decision function auditable without sacrificing the non-linear pattern detection of tree-based methods.

Rules are harvested from an ensemble of gradient boosted trees or random forests. Each node in a tree translates to a conjunctive rule:

• Rule format: IF (feature_A <= threshold) AND (feature_B > value) THEN rule_fires = 1
• Rule depth: Typically limited to 2-4 splits to maintain interpretability
• Rule diversity: Hundreds of candidate rules are generated, then the Lasso selects only the most globally predictive subset This approach captures non-linear interactions and threshold effects that linear models alone miss.

Because the final model is a linear combination, the absolute coefficient value of each rule or feature directly quantifies its contribution to the prediction. This provides:

• Global importance: Which rules and features matter most across all predictions
• Local importance: For any single transaction, the fired rules with non-zero coefficients explain exactly why a specific score was assigned
• Directionality: Positive coefficients push toward fraud, negative toward legitimate This eliminates the need for post-hoc explanation tools like SHAP or LIME for the final model.

To handle outlier transactions common in financial data, RuleFit applies winsorization to original features before rule generation. Extreme values beyond specified percentiles (e.g., 1st and 99th) are capped rather than removed. This prevents:

• Rule fragmentation: Outliers creating overly specific, low-coverage rules
• Coefficient instability: Single extreme transactions distorting the Lasso penalty
• Overfitting: The model memorizing rare anomalies instead of learning generalizable fraud patterns

Unlike traditional regression where interaction terms must be explicitly engineered, RuleFit automatically discovers feature interactions through the tree ensemble. A single rule like IF (transaction_amount > $10,000) AND (account_age_days < 30) captures a two-way interaction. The Lasso then selects only the interactions that improve predictive performance. This is critical for fraud detection where contextual anomalies—combinations of otherwise normal values—are the primary signal.

The final model includes both rule terms and linear terms for original features, allowing it to:

• Capture non-linearities through rules derived from tree splits
• Retain linear relationships where a simple monotonic effect exists (e.g., higher transaction amount linearly increases risk)
• Handle mixed signal types: Rules excel at threshold-based categorical patterns; linear terms handle continuous gradients This dual representation ensures the model is neither unnecessarily complex where a linear relationship suffices nor blind to interactions where they matter.

A game-theoretic framework that assigns each feature an importance value for a particular prediction. SHAP values unify several attribution methods to explain any model's output.

• Key Benefit: Consistent and theoretically grounded feature attribution.
• Fraud Use Case: Explains exactly why a specific transaction was flagged by quantifying the contribution of features like transaction amount, time, and device ID.
• Relation to RuleFit: Both provide interpretable output, but SHAP is a post-hoc method for black-box models, while RuleFit generates an inherently interpretable sparse linear model of rules.

A technique that explains any classifier's prediction by approximating it locally with an interpretable surrogate model (e.g., a linear model) around a specific prediction.

• Mechanism: Perturbs the input instance, observes prediction changes, and fits a simple, explainable model to the local neighborhood.
• Fraud Use Case: Generates human-readable explanations for a single high-risk transaction, showing which features pushed it over the threshold.
• Contrast with RuleFit: LIME provides local fidelity for a single instance; RuleFit provides a global, sparse linear model combining rules and original features.

A glass-box model that learns interpretable feature functions for each input using cyclic gradient boosting. EBM predictions are exact, modular, and human-readable.

• Key Property: The contribution of each feature is an additive shape function that can be plotted and inspected independently.
• Fraud Use Case: Provides a fully auditable model where compliance officers can see the exact effect of a velocity check or a beneficiary mismatch on the final risk score.
• Synergy with RuleFit: Both are inherently interpretable, additive models. RuleFit uses tree-derived rules as features; EBM learns smooth functions per feature.

A method that explains a model's decision by identifying the minimal changes to an input instance's features that would alter the prediction to a desired outcome.

• Example: 'If the transaction amount were $200 lower and the device was previously authenticated, the transaction would not have been flagged.'
• Fraud Use Case: Provides actionable guidance to customers or investigators on how to remediate a blocked transaction.
• Complement to RuleFit: RuleFit's rules define the decision boundary; counterfactuals show the shortest path across that boundary.

A global, model-agnostic visualization showing the marginal effect of one or two features on the predicted outcome, averaged over all other features.

• Insight Provided: Reveals the direction and shape of a feature's relationship with the target (e.g., linear, monotonic, non-linear).
• Fraud Use Case: Visualizes how the probability of fraud changes as the transaction amount increases, holding all else constant on average.
• Limitation: Can be misleading when features are highly correlated, a problem addressed by Accumulated Local Effects (ALE) plots.

A model inspection technique that measures the decrease in model performance when a single feature's values are randomly shuffled, breaking its relationship with the target.

• Interpretation: A large drop in performance indicates high importance; no drop suggests the feature is not used by the model.
• Fraud Use Case: Identifies which signals (e.g., IP geolocation, transaction velocity) are most critical for the model's detection accuracy.
• Contrast with RuleFit: RuleFit performs automatic feature selection via LASSO regularization, while permutation importance is a post-hoc analysis tool.