Fairness Metric (Disparate Impact)

The 80% Rule, established by the U.S. Equal Employment Opportunity Commission (EEOC), is the primary statistical test for disparate impact in employment. It states that a selection rate for any protected group (e.g., race, gender) that is less than 80% (or four-fifths) of the rate for the group with the highest selection rate may constitute evidence of adverse impact.

• Calculation: (Selection Rate for Group A) / (Selection Rate for Group with Highest Rate) >= 0.8
• Example: If an AI resume screener selects 50% of male applicants but only 30% of female applicants, the ratio is 0.6 (30%/50%), which is below the 0.8 threshold, indicating potential disparate impact.
• Purpose: Serves as a bright-line, initial screening tool to flag systems for further legal and technical scrutiny.

Title VII of the Civil Rights Act is the foundational U.S. federal law prohibiting employment discrimination. It is the statutory basis for the disparate impact legal theory.

• Key Precedent: Griggs v. Duke Power Co. (1971) established that employment practices with a disparate impact are illegal, even without proof of discriminatory intent, if they are not job-related and consistent with business necessity.
• Burden-Shifting Framework:
  1. Plaintiff demonstrates a statistically significant disparate impact.
  2. Defendant (Employer) must prove the practice is a business necessity.
  3. Plaintiff can still prevail by showing an available alternative practice exists that serves the same business purpose with less discriminatory effect.
• Application to AI: This framework directly applies to algorithmic hiring, promotion, and credit scoring tools used in employment contexts.

The Equal Credit Opportunity Act (ECOA) and its implementing regulation, Regulation B, prohibit discrimination in any aspect of a credit transaction. It explicitly recognizes the disparate impact theory.

• Regulatory Guidance: The Consumer Financial Protection Bureau (CFPB) has stated that creditors are liable for algorithmic discrimination under ECOA, regardless of intent, if a model results in a disparate impact on a protected class.
• Protected Classes: Includes race, color, religion, national origin, sex, marital status, age, and receipt of public assistance.
• Business Necessity Defense: Similar to Title VII, a creditor may defend a model by demonstrating it is empirically derived, demonstrably and statistically sound, and meets a legitimate business need. The CFPB emphasizes the need for ongoing testing and monitoring for disparate impact.

The EU AI Act takes a risk-based regulatory approach, where high-risk AI systems (including those for employment, credit, and essential services) face stringent requirements. While it does not use the U.S. term "disparate impact," its requirements for fundamental rights impact assessments and bias monitoring serve a parallel purpose.

• High-Risk System Obligations: Providers must implement risk management systems and data governance practices to mitigate risks of algorithmic bias that could lead to discriminatory outcomes.
• Conformity Assessment: Before deployment, high-risk systems must undergo assessment to ensure they do not perpetuate prohibited discrimination.
• Post-Market Monitoring: Continuous logging and monitoring are required to detect any drift in model performance that could create new discriminatory effects.

While not yet law, the proposed Algorithmic Accountability Act represents a forward-looking U.S. legislative framework that would mandate impact assessments for automated systems, explicitly including disparate impact analysis.

• Covered Entities: Would apply to large companies and critical software vendors.
• Required Assessments: Companies would be required to perform impact assessments evaluating their automated systems for impacts on accuracy, fairness, bias, discrimination, privacy, and security.
• Public Reporting: A summary of these assessments, including steps taken to mitigate disparate impact, would need to be published in a publicly accessible repository. This aims to create transparency and accountability for high-impact algorithmic decision-making.

This is the canonical use case for disparate impact analysis, often scrutinized under the Uniform Guidelines on Employee Selection Procedures. Auditors apply the four-fifths rule (80% rule) to automated resume screening, video interview analysis, and skills assessment tools. For example, if a model recommends 50% of male applicants for interviews but only 30% of female applicants, the 60% ratio indicates a potential disparate impact. Analysis focuses on protected attributes like gender, race, and age to ensure selection rates are statistically equitable.

Financial institutions and regulators (e.g., the Consumer Financial Protection Bureau) use disparate impact analysis to audit algorithmic credit models for compliance with the Equal Credit Opportunity Act (ECOA). The test compares approval rates, interest rates, and credit limits across demographic groups defined by race, national origin, or sex. A key challenge is distinguishing between legitimate risk-based pricing and discriminatory proxies, requiring analysis of feature attribution to see if zip code or shopping history acts as a surrogate for a protected class.

Tools like COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) used for predicting recidivism have been extensively audited for disparate impact. Analysis examines whether the models assign higher risk scores to defendants of one race compared to another with similar criminal histories, potentially leading to harsher bail or sentencing recommendations. This domain highlights the tension between predictive parity (equal precision across groups) and demographic parity (equal selection rates), as optimizing for one can violate the other.

Disparate impact analysis is applied to algorithms that predict healthcare needs, prioritize patients for interventions, or aid in clinical diagnosis. For instance, a model used to enroll high-risk patients into care management programs must be audited to ensure it does not systematically exclude groups based on race or socioeconomic status, which could be correlated with training data gaps. This is critical for compliance with Section 1557 of the Affordable Care Act, which prohibits discrimination in health programs.

Platforms use disparate impact analysis to audit ad delivery algorithms for digital redlining, where housing, employment, or credit ads are disproportionately shown or withheld from users based on inferred demographic characteristics. This analysis often involves running controlled experiments (A/A tests) to measure delivery rates across groups when ad content and targeting parameters are held constant, ensuring compliance with laws like the Fair Housing Act.

Educational institutions apply disparate impact analysis to automated systems that pre-screen applications or award scholarships. The four-fifths rule is used to compare recommendation rates across groups based on race, gender, or disability status. The analysis must account for legitimate educational qualifications while ensuring the model does not amplify historical biases present in training data, such as correlations between extracurricular activities and socioeconomic status.

A legal doctrine of fairness focusing on intentional discrimination in decision-making processes. Unlike disparate impact, which examines outcomes, disparate treatment assesses whether a model's algorithm or design explicitly uses a protected attribute (e.g., race, gender) as an input for differential treatment.

• Key Difference: Concerned with process and intent, not just statistical outcomes.
• Example: A loan approval model that directly uses 'zip code' as a feature, where zip code is a proxy for racial demographics, could constitute disparate treatment.
• Legal Test: Requires evidence of discriminatory intent or a facially discriminatory policy.

A group fairness metric that requires a model's predictions to be equally accurate across demographic groups. It mandates that both true positive rates and false positive rates are identical for all protected groups.

• Formal Definition: For all groups a and b, P(Ŷ=1 | Y=1, A=a) = P(Ŷ=1 | Y=1, A=b) AND P(Ŷ=1 | Y=0, A=a) = P(Ŷ=1 | Y=0, A=b).
• Use Case: Critical in high-stakes domains like criminal justice or hiring, where both types of errors (false positives and false negatives) must be fair.
• Trade-off: Often impossible to satisfy simultaneously with other fairness criteria like demographic parity, leading to fairness impossibility theorems.

A group fairness criterion stating that the selection rate (positive prediction rate) should be equal across protected groups. Also known as statistical parity.

• Formula: P(Ŷ=1 | A=a) = P(Ŷ=1 | A=b) for all groups a, b.
• Contrast with Disparate Impact: Demographic parity is the mathematical ideal (a ratio of 1.0), while the 80% rule (disparate impact) is a legal threshold for identifying potential discrimination.
• Criticism: Can conflict with meritocratic principles if base rates of qualification differ between groups. Enforcing it may require quotas or significant model adjustment.

A fairness paradigm that requires similar individuals to receive similar predictions, regardless of their group membership. It contrasts with group fairness metrics like disparate impact.

• Core Principle: "Treat like cases alike."
• Implementation Challenge: Requires defining a meaningful similarity metric for individuals, which is often non-trivial and domain-specific.
• Example: Two loan applicants with identical credit scores, debt-to-income ratios, and employment history should receive the same prediction, even if they belong to different racial groups.

A causality-based fairness definition that evaluates whether a model's prediction for an individual would remain the same if the individual's protected attribute (e.g., race, gender) were changed, while all other relevant, non-discriminatory attributes are held constant.

• Rooted in Causal Models: Requires constructing a causal graph to distinguish discriminatory from legitimate factors.
• Strong Guarantee: Aims to remove the direct and indirect influence of the protected attribute on the prediction.
• Complexity: Demands significant domain knowledge to build the causal model and is computationally intensive to audit.