Algorithmic Fairness

Group fairness metrics quantify equity by comparing statistical outcomes across demographic subgroups defined by protected attributes like race or gender. They provide a mathematical basis for auditing models.

• Demographic Parity: Requires the overall rate of positive predictions (e.g., loan approvals) to be equal across groups.
• Equal Opportunity: Requires the true positive rate (recall) to be equal across groups, ensuring qualified individuals have an equal chance of a favorable outcome.
• Equalized Odds: A stricter criterion requiring both true positive rates and false positive rates to be equal across groups.

These metrics are often in tension with each other and with model accuracy, a fundamental challenge known as the fairness-accuracy trade-off.

Bias can be introduced at multiple stages of the ML lifecycle, leading to two primary legal and technical categories of unfair outcomes.

• Disparate Treatment: Occurs when a model explicitly uses a protected attribute as a direct input to make different decisions for different groups. This is often a result of flawed feature engineering.
• Disparate Impact: Occurs when a model's outputs, while facially neutral, have a disproportionately adverse effect on a protected group. This can be caused by proxy variables (e.g., zip code correlating with race) or biased training data.
• Bias in Data: The root cause often lies in the dataset itself, including historical bias (past societal inequities), representation bias (under-sampling of groups), or measurement bias (flawed data collection).

Technical interventions to reduce unfair discrimination are applied at three key stages of the machine learning pipeline.

• Pre-processing: Techniques applied to the training data before model training. Examples include reweighting samples, transforming features to decorrelate them from protected attributes, or generating synthetic data for underrepresented groups.
• In-processing: Techniques applied during model training by modifying the learning algorithm. This includes adding fairness constraints to the loss function or using adversarial debiasing, where a secondary network tries to predict the protected attribute from the main model's representations.
• Post-processing: Techniques applied to a trained model's predictions. The most common method is adjusting decision thresholds separately for each demographic group to satisfy a target fairness metric like equalized odds, without retraining the model.

Systematic evaluation is required to move from principles to practice. This involves structured assessments and tooling.

• Bias Audit: A systematic, documented evaluation to detect, measure, and report on discriminatory biases against defined protected groups. This is a core component of an Algorithmic Impact Assessment (AIA).
• Subgroup & Intersectional Analysis: Evaluating performance metrics (accuracy, F1) separately for distinct demographic slices. Intersectional analysis examines subgroups at the crossroads of multiple attributes (e.g., Black women), where bias is often compounded.
• Fairness Toolkits: Software libraries like IBM's AI Fairness 360 (AIF360) or Microsoft's Fairlearn provide standardized implementations of metrics, bias detection algorithms, and mitigation techniques for developers.
• Model Cards: Short documents that accompany trained models, transparently reporting performance characteristics, intended use, and known fairness limitations across subgroups.

Beyond group statistics, more nuanced notions of fairness focus on the causal mechanisms of decisions or treat similar individuals similarly.

• Counterfactual Fairness: A causal notion requiring that a model's prediction for an individual would remain the same in a counterfactual world where that individual's protected attribute (e.g., race) had been different, holding all else equal. This relies on constructing a causal model of the data-generating process.
• Individual Fairness: The principle that "similar individuals should receive similar predictions." This requires defining a meaningful similarity metric for individuals within the context of the task, which is often a significant technical challenge.
• Word Embedding Association Test (WEAT): Used to measure implicit societal biases (e.g., gender stereotypes) captured in the geometric relationships between words in a model's embedding space, relevant for auditing bias in Large Language Models (LLMs).

Ensuring fairness is not a one-time task but requires ongoing processes integrated into the ML lifecycle and organizational governance.

• Bias Drift: The phenomenon where a deployed model's fairness performance degrades over time due to changes in the underlying data distribution or societal norms, necessitating continuous monitoring alongside traditional performance drift.
• Proxy Variable Identification: A critical audit step to find features in the data (e.g., occupation, shopping patterns, zip code) that are highly correlated with a protected attribute and could serve as a surrogate for it, enabling disparate impact.
• Fairness-Accuracy Trade-off: The well-documented tension where optimizing strongly for a group fairness metric (like demographic parity) often requires sacrificing some degree of overall predictive accuracy. Managing this trade-off is a key business and technical decision.

Methods applied to the training data before model training to remove underlying biases. The goal is to create a fairer dataset, which serves as the foundation for a fair model.

• Reweighting: Adjusting the weight of individual samples in the training set to balance outcomes across groups.
• Disparate Impact Remover: A technique that edits feature values to reduce discrimination while preserving rank-ordering within groups.
• Learning Fair Representations: Transforming input data into a new representation (latent space) that minimizes information about protected attributes while preserving utility for the prediction task.

Example: In a hiring dataset, reweighting might give more importance to resumes from an underrepresented gender who were historically hired at lower rates, correcting for past bias.

Methods applied during model training by modifying the learning algorithm itself. These techniques directly optimize for both accuracy and fairness.

• Adversarial Debiasing: A primary model is trained to make accurate predictions, while an adversarial model simultaneously tries to predict the protected attribute from the primary model's internal representations. This forces the primary model to learn features uncorrelated with bias.
• Fairness Constraints: Incorporating mathematical conditions like demographic parity or equalized odds directly into the model's loss function as regularization terms.
• Prejudice Remover: Adds a regularization term that penalizes the model for dependence between its predictions and the protected attribute.

Key Advantage: Directly shapes the model's decision boundary, often leading to a better accuracy-fairness trade-off than post-processing.

Methods applied to a trained model's predictions or scores after training. This approach does not require retraining the model, making it useful for auditing and adjusting black-box systems.

• Threshold Adjustment: Applying different decision thresholds for different demographic groups to achieve parity in error rates (e.g., equal opportunity).
• Reject Option Classification: For instances where the model's prediction confidence is low (near the decision boundary), the outcome is assigned to the favorable class for the disadvantaged group.
• Calibrated Equalized Odds Postprocessing: Optimizes over possible derived predictors to satisfy equalized odds with minimal impact on accuracy.

Use Case: A bank could apply different approval thresholds to loan application scores from different zip codes to correct for a biased model, ensuring equal false negative rates.

A specific and powerful in-processing technique that uses a minimax game between two neural networks to remove bias.

Mechanism:

1. A Predictor Network is trained to perform the main task (e.g., credit approval).
2. An Adversary Network is trained to predict the protected attribute (e.g., gender) from the predictor's hidden layers or predictions.
3. The predictor is updated to both minimize its prediction error and maximize the adversary's prediction error. This forces the predictor to learn representations that are useless for discriminating based on the protected attribute.

Outcome: The final model's predictions become decorrelated from the sensitive attribute, promoting individual fairness. Frameworks like TensorFlow's TFCO (TensorFlow Constrained Optimization) provide implementations.

The formal, mathematical approach to enforcing fairness by treating it as a constrained optimization problem. Instead of just minimizing prediction error, the solver must find model parameters that satisfy defined fairness criteria.

Common Constraints:

• Demographic Parity: (Selection Rate) P(Ŷ=1 | A=0) = P(Ŷ=1 | A=1)
• Equal Opportunity: (True Positive Rate) P(Ŷ=1 | A=0, Y=1) = P(Ŷ=1 | A=1, Y=1)
• Equalized Odds: Requires both True Positive Rate and False Positive Rate to be equal across groups.

Implementation: Libraries like Google's TFCO or IBM's AIF360 allow developers to specify these constraints as part of the model's training loop, using techniques like Lagrangian multipliers to handle the trade-offs.

A form of algorithmic bias where a model's outputs, while facially neutral in design, produce a disproportionately adverse effect on members of a protected group. This is a central legal concept in fairness auditing.

• Key Mechanism: The effect is measured by comparing outcome rates (e.g., denial rates) across groups.
• Example: A resume screening tool that rejects a significantly higher percentage of applicants from a particular demographic, even if it doesn't use demographic data directly.
• Legal Basis: Found in regulations like the U.S. Equal Employment Opportunity Commission's 'four-fifths rule' (or 80% rule) for identifying adverse impact.

Quantitative measures used to assess whether a model's predictions are equitable across demographic subgroups. Different metrics formalize competing definitions of 'fairness'.

• Demographic Parity: Requires the overall rate of positive predictions to be equal across groups. Focuses on the outcome.
• Equal Opportunity: Requires the true positive rate (recall) to be equal across groups. Focuses on not missing qualified candidates.
• Equalized Odds: A stricter condition requiring both true positive rates and false positive rates to be equal across groups.
• Trade-offs: These metrics are often mutually exclusive and trade off against overall accuracy, a phenomenon formalized as the fairness-accuracy Pareto frontier.

Technical interventions applied during the ML lifecycle to reduce unfair discrimination. They are categorized by when they are applied.

• Pre-processing: Techniques applied to the training data, such as reweighting samples or transforming features to remove correlations with protected attributes.
• In-processing: Techniques applied during model training, such as adding fairness constraints to the loss function or using adversarial debiasing.
• Post-processing: Techniques applied to the model's predictions after training, such as adjusting decision thresholds separately for each subgroup to meet a fairness goal.

A systematic, documented evaluation of an AI system to detect, measure, and report on potential discriminatory biases. It is a cornerstone of responsible AI governance.

• Process: Involves subgroup analysis to slice performance metrics (accuracy, FPR, FNR) by protected attributes.
• Scope: Examines bias in the training data, the model's predictions, and the real-world impact of its deployment.
• Output: Results are often documented in artifacts like Model Cards or an Algorithmic Impact Assessment (AIA) report for stakeholders and regulators.

A feature in a dataset that is highly correlated with a protected attribute, allowing a model to discriminate indirectly even when the protected attribute is explicitly removed.

• Common Examples: Zip code (correlates with race/income), shopping patterns, university name, or even linguistic patterns in text.
• Challenge: Identifying and handling proxies is difficult because the correlation is often statistical and non-obvious.
• Mitigation: Requires careful feature analysis and causal reasoning. Techniques like adversarial debiasing aim to learn representations that are invariant to these proxies.

The tendency of foundation models to generate outputs that reflect or amplify societal stereotypes, prejudices, and historical inequities present in their massive, web-scale training corpora.

• Manifestations: Can appear as gender/racial stereotypes in generated text, skewed associations in word embeddings, or unequal treatment in question-answering.
• Measurement: Tools like the Word Embedding Association Test (WEAT) quantify stereotype associations in embedding spaces.
• Mitigation: Involves curated data filtering, reinforcement learning from human feedback (RLHF) with fairness guidelines, and targeted adversarial debiasing during fine-tuning.