Disparate Impact Ratio

The foundational threshold for adverse impact established by the U.S. Equal Employment Opportunity Commission (EEOC). A selection rate for a protected group that is less than 80% of the rate for the group with the highest selection rate generally constitutes evidence of adverse impact.

• Calculation: DIR = (Selection Rate of Protected Group) / (Selection Rate of Reference Group)
• Example: If 60% of Group A is approved and 40% of Group B is approved, the DIR is 0.40 / 0.60 = 0.67, which falls below the 0.80 threshold, signaling potential discrimination.
• Context: This is a rule of thumb, not an absolute legal standard, and is used as an initial screening tool.

The ratio is a simple division of probabilities, but its interpretation requires careful context. A value of 1.0 represents perfect parity.

• Formula: DIR = P(favorable_outcome | protected_group) / P(favorable_outcome | reference_group)
• DIR > 1.0: The protected group receives the favorable outcome at a higher rate than the reference group (reverse discrimination risk).
• DIR = 1.0: No statistical disparity between groups.
• DIR < 1.0: The protected group is selected at a lower rate. The magnitude of the disparity must be assessed for practical and statistical significance.

A raw ratio below 0.80 is not sufficient for a legal finding; the disparity must be statistically significant and not due to random chance. This separates a genuine pattern from a small-sample anomaly.

• Fisher's Exact Test: Used to calculate the probability of observing the disparity if no true bias existed.
• Standard Deviation Analysis: A common threshold is that the difference in selection rates must exceed two or three standard deviations.
• Practical Significance: Even a statistically significant result must be evaluated for business relevance. A tiny, consistent disparity in a massive dataset may be statistically significant but operationally trivial.

The definition of the reference group and protected group is a critical design choice that directly impacts the calculated ratio. The reference group is typically the demographic cohort with the highest selection rate.

• Binary Comparison: The classic approach compares a single protected group (e.g., women) against a single reference group (e.g., men).
• Multi-Group Comparison: For non-binary attributes like ethnicity, the ratio is calculated for each group against the group with the maximum selection rate.
• Intersectionality: A simple binary ratio can mask discrimination against subgroups (e.g., women of a specific ethnicity). Advanced analysis requires computing ratios for intersectional segments.

The Disparate Impact Ratio is a univariate, outcome-only metric that does not explain the cause of a disparity. It is a starting point for an audit, not a final verdict.

• Ignoring Input Features: A low DIR does not automatically prove illegal bias. A legitimate, job-related qualification may cause a disparity (a business necessity defense).
• Sample Size Sensitivity: Ratios calculated on small subgroups are highly volatile and can trigger false positives.
• Benchmarking Trap: Achieving a perfect 1.0 ratio can sometimes require explicitly using protected attributes in the model, which creates a disparate treatment risk, creating a legal paradox for fairness engineers.

In MLOps pipelines, the Disparate Impact Ratio is implemented as a continuous monitoring metric to detect model drift with fairness implications.

• Pre-Deployment Testing: Calculate the DIR on a holdout test set before model release to establish a baseline fairness posture.
• Production Monitoring: Recalculate the DIR on live inference logs. A sudden drop in the ratio can indicate concept drift that disproportionately harms a protected group.
• Remediation Triggers: An automated alert is typically configured to fire if the DIR crosses below the 0.80 threshold and the result is statistically significant, triggering a model rollback or human review.

The original Uniform Guidelines benchmark from 1978. A selection rate for a protected group that is less than 80% of the rate for the group with the highest selection rate constitutes evidence of adverse impact.

• Calculation: (Selection Rate Protected Group) / (Selection Rate Reference Group)
• Threshold: A ratio below 0.80 triggers a legal presumption of discrimination.
• Context: This is the historical origin of the Disparate Impact Ratio, primarily used in U.S. employment law.

A stricter fairness metric requiring both equal true positive rates and equal false positive rates across groups.

• True Positive Rate Parity: P(ŷ=1 | Y=1, Group A) = P(ŷ=1 | Y=1, Group B)
• False Positive Rate Parity: P(ŷ=1 | Y=0, Group A) = P(ŷ=1 | Y=0, Group B)
• Advantage: Matches the model's error profile across groups, ensuring no group bears a disproportionate share of mistakes.

A fairness definition that allows for legitimate differentiation by controlling for a set of legitimate risk factors (L).

• Definition: P(ŷ=1 | L=l, Group A) = P(ŷ=1 | L=l, Group B)
• Use Case: Accepts that loan default rates may differ by credit score, but demands equal treatment for individuals with the same credit score.
• Challenge: Requires careful selection of the legitimate factor set to avoid masking discrimination.

Determining whether an observed Disparate Impact Ratio is a statistically significant deviation or merely the result of random sampling noise.

• Fisher's Exact Test: Commonly used for small sample sizes and binary outcomes.
• Confidence Intervals: A 95% CI around the ratio provides a range of plausible true values.
• Practical Significance: A ratio of 0.79 might be statistically insignificant with a small sample, but still legally risky.