Disparate Impact

The primary quantitative benchmark established by the EEOC's Uniform Guidelines. A selection rate for a protected group that is less than 80% of the rate for the group with the highest selection rate constitutes evidence of adverse impact.

• Calculation: (Selection Rate of Protected Group) / (Selection Rate of Highest Group)
• Example: If a hiring algorithm advances 60% of male applicants but only 30% of female applicants, the ratio is 30/60 = 0.50, violating the 80% rule.
• Threshold: A ratio below 0.80 triggers a prima facie case of discrimination, shifting the burden of proof to the employer.

Beyond the 80% rule, courts and regulators increasingly rely on formal statistical tests to determine if a disparity is due to chance. The null hypothesis assumes no relationship between protected status and outcome.

• Standard Deviation Analysis: A disparity greater than 2 or 3 standard deviations from the mean is typically considered statistically significant.
• Fisher's Exact Test: Used for small sample sizes to calculate the exact probability of observing a disparity.
• Practical vs. Statistical Significance: A large dataset may show a statistically significant but practically trivial disparity. Both dimensions must be evaluated.

These are two distinct theories of discrimination under U.S. law. The critical distinction is intent.

• Disparate Treatment: Intentional discrimination where an individual is treated differently explicitly because of a protected characteristic. Requires proof of discriminatory motive.
• Disparate Impact: Unintentional discrimination arising from a neutral policy. No proof of intent is required; the focus is solely on the consequences of the practice.
• Algorithmic Context: A model that explicitly uses race as a feature exhibits disparate treatment. A model using a proxy like ZIP code that causes racial disparity exhibits disparate impact.

Once a plaintiff establishes a prima facie case of disparate impact, the burden shifts to the defendant to prove the challenged practice is job-related and consistent with business necessity.

• Validation Required: The employer must demonstrate through empirical evidence that the selection procedure is predictive of or significantly correlated with important elements of job performance.
• No Alternative Practice: Even if business necessity is proven, the plaintiff can still prevail by showing a less discriminatory alternative practice exists that serves the employer's legitimate interests equally well.
• AI Context: A high-performing credit model that disparately impacts a minority group must be demonstrably more accurate than a fairer alternative to survive this defense.

Disparate impact often occurs through proxy variables—seemingly neutral features that are highly correlated with protected attributes, effectively reconstructing the forbidden characteristic.

• Redlining Legacy: Historical housing discrimination makes ZIP code a strong proxy for race in financial models.
• Digital Proxies: Browsing history, purchase patterns, and device type can inadvertently encode socioeconomic status, gender, or age.
• Mitigation: Requires rigorous feature auditing to identify and remove variables with high mutual information with sensitive attributes, or applying fairness constraints during training.

Disparate impact is codified in multiple federal statutes and enforced by specific regulatory bodies.

• Title VII of the Civil Rights Act of 1964: Prohibits employment discrimination based on race, color, religion, sex, and national origin.
• Fair Housing Act: Applies disparate impact analysis to housing and lending practices.
• ECOA (Equal Credit Opportunity Act): Prohibits credit discrimination, enforced by the CFPB.
• EEOC: The primary federal agency responsible for enforcing employment anti-discrimination laws and issuing guidance like the Uniform Guidelines on Employee Selection Procedures.

The primary quantitative test for disparate impact established by the EEOC's Uniform Guidelines. A selection rate for a protected group that is less than 80% of the rate for the group with the highest selection rate constitutes evidence of adverse impact.

• Formula: (Selection Rate of Protected Group) / (Selection Rate of Reference Group) < 0.80
• Example: If 60% of male applicants receive a loan offer but only 30% of female applicants do, the ratio is 0.50—a clear violation.
• This is a threshold test, not a definitive finding of discrimination; it shifts the burden of proof to the employer or algorithm deployer to demonstrate business necessity.

A critical legal distinction in anti-discrimination law. Disparate treatment is intentional discrimination where a protected characteristic is explicitly a factor in a decision. Disparate impact involves facially neutral policies or algorithms that disproportionately harm a protected group, regardless of intent.

• Disparate Treatment: A model that explicitly uses race as a feature to deny loans.
• Disparate Impact: A credit model that uses zip code as a proxy feature, unintentionally redlining minority neighborhoods.
• In AI systems, disparate impact is the more common and insidious risk, as models can learn latent proxies for sensitive attributes from seemingly innocuous data.

The legal rebuttal to a prima facie case of disparate impact. An employer or algorithm deployer must demonstrate that the challenged practice is job-related for the position in question and consistent with business necessity.

• The practice must be predictive of or significantly correlated with important elements of work behavior or business outcomes.
• Even if business necessity is proven, the plaintiff can still prevail by showing a less discriminatory alternative exists that serves the same business need without the adverse impact.
• In ML terms, this requires demonstrating that a protected attribute or its proxy is genuinely required for predictive accuracy and no debiased model achieves comparable performance.

Legally defined groups shielded from discrimination. In the US, federal protected classes include race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age (40+), disability, and genetic information. State and local laws may add protections for marital status, veteran status, and other characteristics.

• In AI fairness, these are called sensitive attributes—features that should not influence outcomes.
• Proxy variables (e.g., zip code for race, purchase history for gender) can inadvertently encode sensitive attributes.
• GDPR Article 9 extends special protections to processing data revealing racial or ethnic origin, political opinions, religious beliefs, and biometric data.

The formal methodology for quantifying disparate impact in algorithmic systems. The Adverse Impact Ratio (AIR) compares the positive outcome rate of a protected group to that of a privileged reference group.

• Step 1: Calculate the selection rate for each group (e.g., percentage receiving a recommendation, approval, or high score).
• Step 2: Identify the group with the highest selection rate as the reference.
• Step 3: Compute AIR = (Protected Group Rate) / (Reference Group Rate).
• Step 4: Flag if AIR < 0.80 for further investigation.
• Modern fairness toolkits like AI Fairness 360 and Fairlearn automate this calculation across multiple groups and thresholds.

Algorithmic interventions to mitigate disparate impact without explicitly using sensitive attributes. These operate at different stages of the ML pipeline:

• Pre-processing: Reweighting or resampling training data to balance representation. Disparate impact remover edits feature values to reduce dependence on protected attributes while preserving rank-ordering within groups.
• In-processing: Adding fairness constraints to the model's objective function. Adversarial debiasing trains a predictor while an adversary tries to infer the protected attribute from predictions.
• Post-processing: Adjusting decision thresholds per group to equalize outcomes. Reject option classification gives favorable outcomes to unprivileged groups near the decision boundary.