Fairness Metric (Disparate Impact)

Disparate Impact functions as a statistical test for discrimination, focusing solely on outcomes. It does not require proof of discriminatory intent, making it a cornerstone of disparate impact theory in law and algorithmic auditing. The core calculation is a simple ratio:

• Formula: (Selection Rate for Unprivileged Group) / (Selection Rate for Privileged Group)
• A result of 1.0 indicates perfect parity.
• The widely cited 80% Rule (or four-fifths rule) from U.S. Equal Employment Opportunity Commission guidelines suggests a ratio below 0.8 may indicate adverse impact warranting investigation.

This metric is a primary measure of group fairness (also called statistical parity or demographic parity). It evaluates fairness at the population level by comparing aggregate outcomes for predefined groups (e.g., based on race, gender, age).

Key Implications:

• It does not assess individual fairness, which considers whether similar individuals receive similar outcomes.
• Achieving a Disparate Impact ratio of 1.0 may conflict with other fairness definitions or accuracy metrics, leading to the fairness-accuracy trade-off.
• It is most applicable when the selection process should be blind to the protected attribute.

The metric is directly rooted in anti-discrimination law, particularly U.S. employment law (Title VII of the Civil Rights Act). It provides a quantitative method for enforcing legal standards against practices that are fair in form but discriminatory in operation.

Regulatory Context:

• Used by the EEOC and OFCCP in compliance evaluations.
• Influences standards in fair lending (Equal Credit Opportunity Act).
• Informs emerging regulations like the European Union AI Act, which mandates assessment of "prohibited discrimination" through such statistical measures.

Disparate Impact is inherently threshold-dependent. The calculated ratio can change dramatically based on the classification threshold used to binarize model scores into positive/negative decisions (e.g., "hire" or "deny loan").

Engineering Consideration:

• A model may show high Disparate Impact at one threshold (e.g., 0.5) but not at another (e.g., 0.7).
• This necessitates analysis across the full range of thresholds, often visualized alongside the ROC curve or precision-recall curve.
• Mitigation strategies often involve threshold adjustment for different groups to achieve parity, a technique known as equalized odds post-processing.

It is crucial to distinguish Disparate Impact from Disparate Treatment.

Disparate Treatment is intentional discrimination where a protected attribute is explicitly used in decision-making. Disparate Impact is unintentional discrimination arising from a facially neutral policy that disproportionately harms a protected group.

In machine learning:

• Disparate Treatment could occur if a protected attribute (e.g., 'race') is used directly as a model feature.
• Disparate Impact can occur even when protected attributes are excluded, if the model learns proxies for them from other correlated features (e.g., 'zip code' proxying for race).

While foundational, Disparate Impact has well-documented limitations that ML engineers must consider:

• Simpson's Paradox: Group-level parity can mask discrimination within subgroups.
• Base Rate Ignorance: It does not account for legitimate differences in qualification rates between groups, potentially forcing quotas.
• Causal Ambiguity: A low ratio indicates a disparity but does not prove the model is the cause; it may reflect historical biases in the training data.
• Multiple Groups: Applying the 80% rule pairwise across many groups can lead to conflicting requirements. It is often supplemented with metrics like Statistical Parity Difference or analyses using Causal Inference frameworks.

Disparate Impact is a primary metric for auditing automated hiring tools. Regulators and internal compliance teams calculate the ratio of candidates recommended for interviews from different demographic groups (e.g., by gender or ethnicity).

• Key Calculation: The ratio of selection rates (e.g., "call-back" rates) for an unprivileged group versus a privileged group.
• Legal Threshold: A ratio below 0.8 (or 80%) often indicates adverse impact under U.S. Equal Employment Opportunity Commission guidelines, triggering a legal review.
• Example: If an AI resume screener selects 10% of male applicants and 4% of female applicants for interviews, the disparate impact ratio is 0.4 (4%/10%), signaling severe potential bias.

Financial institutions and regulators use Disparate Impact to ensure algorithmic credit models do not unfairly disadvantage protected classes, such as certain racial groups.

• Application: Comparing the approval rate for loan applications across ZIP codes or demographic categories.
• Regulatory Context: This metric is central to enforcement of the U.S. Equal Credit Opportunity Act (ECOA).
• Real-World Focus: A finding of disparate impact does not prove intentional discrimination but places the burden on the lender to demonstrate the model's factors are a "business necessity" and no less discriminatory alternative exists.

In criminal justice, Disparate Impact analysis scrutinizes tools used for predictive policing (where to patrol) or recidivism risk scoring (e.g., COMPAS).

• Core Issue: These systems often show high disparate impact, flagging individuals from historically over-policed communities at higher rates.
• Metric Role: It quantifies the disparity in "positive" (high-risk) predictions between racial groups, raising ethical and legal questions about reinforcing systemic biases.
• Critical Limitation: A low disparate impact ratio here may still mask label bias, if the historical arrest data used for training is itself biased.

Disparate Impact is used to audit clinical AI models to prevent inequitable access to care or diagnostic resources.

• Use Case 1: Analyzing an algorithm that identifies patients for high-risk care management programs. A disparate impact finding might show elderly or low-income patients are under-referred.
• Use Case 2: Evaluating computer-aided diagnosis tools for skin cancer, ensuring they perform equally well across skin tones.
• Importance: Bias here can directly affect patient outcomes and violate principles of equitable care.

Platforms audit their ad delivery algorithms for Disparate Impact to prevent discriminatory outcomes, such as showing high-paying job ads only to male users or certain housing ads only to specific racial groups.

• Mechanism: Even if an advertiser targets a broad audience, the platform's optimization algorithm (aiming for clicks) can learn and replicate societal biases in delivery.
• Audit Process: Researchers measure the rate at which different demographic groups are shown a particular ad category.
• Legal Implication: This can lead to lawsuits under civil rights laws regarding housing and employment advertising.

Educational institutions may use Disparate Impact to proactively evaluate automated tools for processing applications or awarding scholarships.

• Objective: To ensure algorithms do not inadvertently disadvantage applicants based on protected attributes like nationality, gender, or socioeconomic background inferred from data.
• Proactive Compliance: Calculating the ratio of admission/scholarship recommendations across groups helps institutions meet diversity goals and avoid legal challenges.
• Complexity: Must be balanced with other lawful institutional goals, making it a tool for diagnosis and transparency rather than a sole decision rule.

Disparate Treatment refers to intentional, explicit discrimination where a model's algorithm or decision rule treats individuals differently based on their membership in a protected class (e.g., race, gender). Unlike Disparate Impact, which examines outcomes, this focuses on discriminatory inputs or processes.

• Key Difference: Looks at intent or explicit use of protected attributes.
• Example: A loan model that directly uses 'zip code' as a feature, which acts as a proxy for race, resulting in different scoring rules.
• Legal Context: Often easier to prove in court as it requires evidence of discriminatory intent in the model's design.

The Equal Opportunity Difference is a fairness metric that compares the true positive rates (recall) between an unprivileged group and a privileged group. A value of zero indicates perfect equality of opportunity.

• Calculation: TPR_unprivileged - TPR_privileged
• Interpretation: Measures if the model is equally good at identifying positive outcomes for all groups. A negative value indicates the model has lower recall for the unprivileged group.
• Use Case: Critical in applications like hiring or lending, where missing a qualified candidate (a false negative) is a significant harm.
• Relation to Disparate Impact: Disparate Impact looks at overall positive outcome rates; Equal Opportunity Difference focuses specifically on the model's performance on actual positive cases.

Statistical Parity Difference is a core group fairness metric that measures the difference in the probability of receiving a favorable outcome between groups. It is directly related to the Disparate Impact ratio.

• Calculation: P(Ž=1 | D=unprivileged) - P(Ž=1 | D=privileged) where Ž is the model's prediction and D is the group attribute.
• Interpretation: A value of 0 indicates perfect statistical parity. This metric aligns with the 80% rule (Disparate Impact): a ratio of probabilities below 0.8 often corresponds to a Statistical Parity Difference more negative than -0.2.
• Limitation: Enforcing a SPD of zero may require sacrificing model accuracy, as it ignores differences in base rates or qualifications between groups.

The Average Odds Difference is a fairness metric that averages the difference in false positive rates and the difference in true positive rates between groups. It enforces both equal opportunity and equal false positive rates.

• Calculation: 1/2 * [(FPR_unprivileged - FPR_privileged) + (TPR_unprivileged - TPR_privileged)]
• Interpretation: A value of zero indicates the model has equal odds across groups. It is a stricter criterion than Equal Opportunity alone.
• Context: Useful in criminal justice risk assessments, where both falsely labeling a low-risk person as high-risk (FPR) and failing to identify a high-risk person (TPR) carry serious consequences.
• Trade-off: Satisfying this constraint often requires significant trade-offs with overall model accuracy.

The Theil Index is an inequality metric borrowed from economics, adapted to measure fairness in machine learning by quantifying the disparity in model performance or outcomes across subgroups.

• Basis: Measures entropy or inequality in the distribution of a metric (e.g., accuracy, positive rate) across multiple groups.
• Advantage: Can handle more than two groups simultaneously, unlike metrics like Statistical Parity Difference.
• Interpretation: A value of 0 represents perfect equality. Higher values indicate greater inequality in outcomes.
• Application: Used in comprehensive fairness audits to get a single, aggregate measure of disparity across many protected attributes (e.g., intersecting race, gender, and age categories).

Counterfactual Fairness is a causal fairness notion that asks: "Would the model's prediction have been the same for an individual if their protected attribute (e.g., race) were different, while all other relevant, non-discriminatory factors remained the same?"

• Causal Approach: Requires modeling the underlying causal relationships between variables, not just observing correlations.
• Strength: Aims to remove the influence of discriminatory paths in the causal graph, offering a more nuanced view than statistical parity.
• Implementation Challenge: Requires a specified causal model, which can be difficult to construct and validate from observational data.
• Contrast with Disparate Impact: Disparate Impact is a purely observational, outcome-based test. Counterfactual Fairness seeks to understand and correct the mechanisms that lead to disparate outcomes.