Post-hoc Explainability

Post-hoc methods are designed to be model-agnostic, meaning they can explain the predictions of any machine learning model—from gradient-boosted trees to deep neural networks—without requiring access to the model's internal architecture. This is critical in fraud detection, where ensembles and proprietary models are common.

• Treats the model as a black box, analyzing only inputs and outputs
• Enables a unified explanation framework across heterogeneous model inventories
• Contrasts with intrinsic interpretability, which requires a specific model structure like linear regression or a single decision tree

Post-hoc explainability operates at two distinct scopes. Local explanations justify a single prediction—such as why a specific transaction was flagged as fraudulent—using methods like LIME or SHAP. Global explanations reveal the model's overall behavior across all predictions, often visualized through Partial Dependence Plots or surrogate models.

• Local: Generates reason codes for adverse actions, satisfying regulatory requirements like the FCRA
• Global: Identifies systematic biases or over-reliance on specific features
• Both scopes are necessary for a complete algorithmic audit trail

The core mechanism of most post-hoc techniques is feature attribution, which quantifies each input variable's contribution to a specific prediction. Methods differ in their mathematical rigor:

• SHAP uses Shapley values from cooperative game theory to guarantee a fair, additive distribution of importance
• Integrated Gradients computes path integrals of gradients for deep networks, satisfying the completeness axiom
• LIME creates a local, interpretable surrogate model via perturbation sampling
• These attributions directly translate into customer-facing adverse action reason codes

Beyond feature importance, post-hoc explainability includes counterfactual explanations that identify the minimal set of changes required to flip a model's decision. For a denied transaction, a counterfactual might state: 'If the transaction amount were below $5,000 and the merchant category were not electronics, the transaction would have been approved.'

• Provides actionable guidance for remediation
• Aligns with human reasoning about causality and 'what-if' scenarios
• Complements adverse action reason codes with prescriptive, not just descriptive, information

Advanced post-hoc methods like TCAV (Testing with Concept Activation Vectors) move beyond low-level feature attribution to explain predictions in terms of high-level, human-understandable concepts. In fraud detection, this means explaining a decision not by raw pixel values or transaction amounts, but by concepts like 'unusual merchant category' or 'velocity anomaly.'

• Uses concept activation vectors to measure sensitivity to predefined ideas
• Bridges the gap between technical model internals and business logic
• Related to Concept Bottleneck Models, which enforce concept-based reasoning during training

For models processing structured or unstructured data, post-hoc explainability produces saliency maps and other visualizations that highlight influential inputs. In fraud detection, this can mean highlighting specific cells in a transaction record or nodes in a financial graph.

• Grad-CAM produces coarse localization maps for convolutional networks
• Layer-wise Relevance Propagation (LRP) decomposes output scores backward through the network with conservation rules
• Individual Conditional Expectation (ICE) Plots disaggregate average effects to reveal heterogeneous relationships across instances