Disparate Impact Testing

The foundational quantitative metric defined by the 80% rule (or four-fifths rule). It is calculated as the selection rate for a protected group divided by the selection rate for the reference group. An AIR below 0.80 constitutes prima facie evidence of disparate impact, triggering deeper legal and statistical scrutiny. This ratio is the primary red-flag indicator in fair lending and employment contexts.

The critical pre-analysis phase of defining and segmenting the population into protected groups based on legally recognized characteristics. In financial contexts, this includes race, ethnicity, sex, age, and marital status. The challenge lies in proxy variable detection—identifying seemingly neutral features like ZIP code or device type that act as statistically significant proxies for protected class membership.

A technique that isolates the impact of a single feature by holding all other variables constant. It answers the question: 'If we change only this feature's value while keeping everything else identical, how does the outcome probability shift for different groups?' This decomposes the aggregate disparity into feature-level contributions, pinpointing the specific variables driving adverse impact.

Beyond the raw ratio, rigorous testing requires establishing that the observed disparity is not due to random chance. Common methods include:

• Fisher's Exact Test: For small sample sizes
• Two-sample Z-test for proportions: For large-scale transaction data
• Bootstrap confidence intervals: To quantify the uncertainty around the AIR estimate A result must be both practically significant (AIR < 0.80) and statistically significant (p < 0.05).

The analytical framework for evaluating whether a feature causing disparate impact is job-related and consistent with business necessity. This involves demonstrating a clear, empirical link between the challenged variable and a legitimate business outcome—such as fraud prediction accuracy—and proving that no less discriminatory alternative (LDA) exists that would serve the same business need with reduced adverse impact.

The systematic process of searching for alternative models, features, or decision rules that achieve comparable business performance while reducing the adverse impact ratio. Techniques include:

• Feature substitution: Replacing a proxy variable with a direct, non-discriminatory measure
• Threshold adjustment: Modifying decision cutoffs for protected groups
• Adversarial debiasing: Training models to minimize protected group information leakage An LDA must be adopted if it satisfies the business objective.

A statistical evaluation of model outcomes across protected demographic classes to detect and remediate disparate treatment or disparate impact. This analysis is mandated by the Equal Credit Opportunity Act (ECOA) and Fair Housing Act, requiring lenders to demonstrate that automated underwriting or fraud scoring models do not systematically disadvantage groups based on race, gender, age, or other protected characteristics. Key techniques include proxy detection for identifying non-obvious correlates of protected class membership and marginal effect analysis to isolate the impact of individual features.

The primary quantitative metric used in disparate impact testing, calculated as the selection rate for a protected group divided by the selection rate for the control group. The Uniform Guidelines on Employee Selection Procedures established the four-fifths rule (80% rule): a ratio below 0.80 constitutes prima facie evidence of adverse impact. In fraud detection, this measures whether legitimate transactions from protected groups are flagged at disproportionately higher rates. A ratio of 0.65, for example, indicates the protected group faces a 35% higher false positive rate.

The process of identifying model features that act as statistical proxies for protected class membership even when explicit demographic attributes are excluded. Common proxies in financial models include:

• ZIP codes correlating with racial composition
• Educational attainment correlating with age
• Credit product types correlating with socioeconomic status Detection methods include mutual information analysis, correlation matrices, and causal inference techniques to determine whether a feature's predictive power derives primarily from its correlation with a protected attribute.

A legal doctrine allowing a facially neutral practice with adverse impact to remain in use if the institution can demonstrate it is job-related and consistent with business necessity. In fraud modeling, this requires proving that a challenged feature is predictively essential for detecting genuine fraud and that no less discriminatory alternative exists. The burden of proof shifts: once adverse impact is established, the institution must validate that removing or replacing the feature would materially degrade fraud detection efficacy and increase financial losses.

A systematic methodology for identifying alternative model features or decision rules that achieve comparable business outcomes with reduced adverse impact. The process involves:

• Feature ablation studies to measure each variable's marginal contribution to both accuracy and disparity
• Constrained optimization to find Pareto-optimal model configurations
• Subgroup reweighting to balance error rates across populations Regulators expect documented evidence that an exhaustive LDA search was conducted before finalizing any model exhibiting adverse impact ratios below the 80% threshold.

The application of SHapley Additive exPlanations (SHAP) to decompose model predictions and quantify each feature's contribution to outcome disparities across groups. By aggregating SHAP values by protected class, practitioners can identify which features drive differential treatment. SHAP dependence plots visualize how feature values interact with protected attributes, revealing whether a model uses legitimate risk factors differently for different groups. This technique bridges the gap between global fairness metrics and local explainability for individual adverse decisions.