Disparate Impact

The primary quantitative benchmark for identifying disparate impact, originating from the Uniform Guidelines on Employee Selection Procedures (1978). A selection rate for a protected group that is less than 80% of the rate for the group with the highest selection rate is evidence of adverse impact.

• Formula: (Selection Rate of Protected Group) / (Selection Rate of Highest Group) < 0.80
• Example: If a hiring algorithm advances 60% of male applicants but only 30% of female applicants, the ratio is 0.50, signaling a potential violation.
• Statistical Significance: Courts may also require that the disparity is statistically significant, often at the p < 0.05 level.

Once a plaintiff establishes a prima facie case of disparate impact, the burden shifts to the defendant to prove the challenged practice is job-related and consistent with business necessity. The practice must be predictive of or significantly correlated with important elements of work behavior.

• Validation: Employers must demonstrate the practice is properly validated, often through content, criterion, or construct validity studies.
• Title VII: Codified in the Civil Rights Act of 1991, which explicitly placed the burden of proof for business necessity on the employer.

Even if an employer proves business necessity, a plaintiff can still prevail by demonstrating that a less discriminatory alternative exists that serves the employer's legitimate interest equally well but with a less severe disparate impact.

• Refusal to Adopt: The employer's refusal to adopt such an alternative is treated as a violation.
• Algorithmic Context: In ML, this often involves re-training a model with a fairness constraint or using a different feature set that achieves comparable accuracy with reduced group disparity.

A critical legal distinction separates these two forms of discrimination. Disparate treatment is intentional discrimination—an employer knowingly treats an individual differently because of a protected characteristic.

• Disparate Impact: Unintentional; a neutral policy has a disproportionately adverse effect.
• Proof of Intent: Disparate impact does not require proof of discriminatory motive. It is an effects-based doctrine.
• Legal Origin: Disparate impact was established by the Supreme Court in Griggs v. Duke Power Co. (1971).

A model can cause disparate impact even without direct access to a protected attribute like race or gender. This occurs through proxy discrimination, where a non-protected feature serves as a stand-in.

• Classic Proxy: Zip code or neighborhood strongly correlates with race due to historical housing segregation.
• Algorithmic Redlining: Using a proxy like browser history or purchasing patterns to offer different credit terms to different groups.
• Detection: Requires auditing feature importance and correlation matrices between model inputs and protected attributes.

Beyond the 80% rule, modern fairness toolkits implement statistical measures to quantify disparate impact in classification models.

• Demographic Parity Difference: The difference in positive prediction rates between groups. A value of 0 indicates perfect parity.
• Disparate Impact Ratio: The ratio of positive prediction rates. A value of 1.0 indicates no disparate impact.
• Tooling: Libraries like Fairlearn, AIF360, and TensorFlow Fairness Indicators compute these metrics directly from model predictions and sensitive attribute data.

The intentional and explicit differential treatment of individuals based on their membership in a protected class. Unlike disparate impact, which concerns facially neutral practices with discriminatory effects, disparate treatment requires proof of discriminatory intent. In machine learning systems, this can manifest when a model is given direct access to a protected attribute like race or gender and uses it to make decisions. This is the most straightforward form of discrimination under U.S. law and is almost always illegal.

The primary quantitative benchmark for identifying disparate impact, established by the Uniform Guidelines on Employee Selection Procedures (1978). A selection rate for a protected group that is less than 80% of the rate for the group with the highest selection rate constitutes evidence of adverse impact.

• Formula: (Selection Rate of Protected Group) / (Selection Rate of Reference Group) < 0.80
• Example: If 60% of male applicants are approved and only 30% of female applicants are approved, the ratio is 0.50, signaling a potential violation.
• Limitation: This is a rule of thumb, not a statistical significance test. It can be unreliable with very small sample sizes.

A form of algorithmic bias where a facially neutral feature serves as a stand-in for a protected attribute, allowing disparate impact to occur indirectly. Common proxies include:

• Zip code as a proxy for race due to residential segregation.
• Credit history as a proxy for socioeconomic status.
• Name or linguistic patterns as proxies for ethnicity or gender.

Simply removing protected attributes from a model—a naive approach called fairness through unawareness—is insufficient because machine learning models can reconstruct sensitive information from correlated proxies.

An independence-based fairness criterion requiring that a model's positive prediction rate be equal across all groups defined by a protected attribute. Also known as statistical parity.

• Formal Definition: P(Ŷ=1 | A=a) = P(Ŷ=1 | A=b) for all groups a, b.
• Key Trade-off: This criterion ignores the ground truth labels. If base rates of a qualified outcome differ between groups, enforcing demographic parity may require selecting unqualified individuals from one group while rejecting qualified individuals from another.
• Relationship to Disparate Impact: Demographic parity is the mathematical formalization of the outcome sought by disparate impact doctrine.

A separation-based fairness metric requiring a classifier to have equal true positive rates and equal false positive rates across different sensitive groups. This is stricter than equal opportunity, which only constrains true positive rates.

• Formal Definition: P(Ŷ=1 | Y=y, A=a) = P(Ŷ=1 | Y=y, A=b) for y ∈ {0,1}
• Implication: The model's errors must be distributed equally across groups. A qualified individual has the same chance of being selected regardless of group membership, and an unqualified individual has the same chance of being erroneously selected.
• Use Case: Preferred in high-stakes domains like criminal justice where both false positives and false negatives carry severe consequences.

The ability for an individual negatively affected by an algorithmic decision to understand the reasons and take actionable steps to reverse that decision in the future. This concept is directly tied to disparate impact remediation.

• Key Components:
  • Actionability: The recommended changes must be under the individual's control (e.g., 'increase income by 20%' is not actionable; 'reduce credit utilization below 30%' is).
  • Counterfactual Generation: Systems like FACE or DiCE generate minimal changes to input features that would flip a negative decision to positive.
• Legal Context: The EU's GDPR Article 22 grants a right to meaningful information about the logic involved in automated decisions, making recourse a regulatory requirement.