Bias Mitigation

Techniques applied to the training dataset before model training to reduce bias at its source.

• Reweighting: Adjusting sample weights to balance representation across demographic groups.
• Resampling: Oversampling underrepresented groups or undersampling overrepresented ones.
• Label Correction: Identifying and correcting biased labels in historical data.
• Fairness-aware Data Augmentation: Generating synthetic data to improve representation of minority groups.

Example: In a hiring dataset, resampling to ensure equal representation of candidates from all educational backgrounds.

Modifying the learning algorithm itself to incorporate fairness as an objective or constraint during training.

• Adversarial Debiasing: Training the main model alongside an adversary that tries to predict a protected attribute (e.g., gender) from the model's embeddings, forcing the main model to learn representations that are invariant to that attribute.
• Regularization for Fairness: Adding a penalty term to the loss function that measures a statistical fairness metric (e.g., demographic parity, equalized odds).
• Constrained Optimization: Formulating training as an optimization problem where accuracy is maximized subject to fairness constraints.

Key Constraint: Demographic Parity requires prediction outcomes to be statistically independent of protected attributes.

Adjusting model predictions after training to satisfy fairness criteria, without retraining the model.

• Threshold Optimization: Applying different decision thresholds for different demographic groups to achieve equal false positive/negative rates.
• Label Flipping: Selectively changing a subset of predicted labels to improve group fairness metrics.
• Rejection Option Classification: Abstaining from making a prediction for instances where the model's confidence is low and the potential for biased error is high.

Primary Use: When you have a pre-trained 'black box' model and need to deploy it with immediate fairness guarantees. It directly modifies the decision rule.

Learning feature representations that are explicitly purged of sensitive information or that encode it in a controlled manner.

• Invariant Representation Learning: Learning a feature space where the distributions of embeddings are identical across different protected groups.
• Counterfactual Fairness: Ensuring a model's prediction for an individual is the same in the actual world and a counterfactual world where the individual belonged to a different demographic group.
• Concept Erasure: Using techniques like Iterative Nullspace Projection (INLP) to linearly remove concepts associated with protected attributes from neural network representations.

Core Mechanism: Projects data into a debiased latent space before final classification or regression.

Using causal graphs and models to distinguish between discriminatory bias and legitimate statistical associations.

• Causal Graphs: Explicitly modeling relationships between protected attributes (A), confounding variables (C), legitimate features (X), and outcomes (Y).
• Intervention vs. Observation: Fairness is assessed based on the model's behavior under an intervention on the protected attribute, not its correlation in observational data.
• Path-Specific Effects: Identifying and blocking discriminatory causal paths (e.g., A → Y) while allowing fair paths (e.g., A → X → Y, if X is a legitimate qualification).

Key Benefit: Provides a principled framework to decide what should be controlled for, avoiding the removal of fair, predictive correlations.

Ongoing measurement of model performance and fairness metrics in production to detect drift and new biases.

• Disparity Tracking: Continuously computing fairness metrics (e.g., disparate impact ratio, average odds difference) across key user segments.
• Sliced Analysis: Evaluating model performance not just on aggregate, but on hundreds of predefined or automatically discovered data slices to find underperforming subgroups.
• Bias Drift Detection: Setting statistical control limits on fairness metrics to trigger alerts when bias exceeds acceptable thresholds.
• Audit Trail Generation: Logging inputs, outputs, and contextual data to enable retrospective bias investigation.

Essential Practice: Bias is not solved once at training; it must be managed as a continuous risk in dynamic systems.

A fairness constraint is a mathematical or programmatic rule applied during AI model training or inference to enforce statistical fairness metrics. These constraints are a direct implementation of bias mitigation goals.

• Types: Includes demographic parity, equality of opportunity, and counterfactual fairness.
• Implementation: Often integrated as regularization terms in the loss function or as post-processing rules on model outputs.
• Example: A loan approval model might be constrained to ensure approval rates do not statistically differ across protected demographic groups, given equal creditworthiness.

Harmful concept erasure is a fine-tuning or model editing technique aimed at removing or neutralizing specific dangerous knowledge or behavioral tendencies from a neural network's weights. It directly mitigates bias by targeting undesirable associations.

• Mechanism: Techniques like weight editing or concept ablation selectively modify model parameters to reduce the probability of generating outputs related to a harmful concept.
• Target: Can erase stereotypes, toxic language patterns, or instructions for illegal activities.
• Challenge: Must be performed without catastrophically degrading the model's general knowledge and reasoning capabilities.

Controlled generation refers to techniques that manipulate a language model's internal representations during inference to guide outputs toward or away from specific attributes. It provides runtime bias mitigation.

• Methods: Includes steering vectors, activation engineering, and guided decoding.
• Use Case: Can dynamically increase the positivity of sentiment or reduce gendered language in real-time without retraining the model.
• Advantage: Offers flexible, adjustable control over model behavior post-deployment, allowing for rapid response to newly identified biases.

Value alignment is the field of AI safety focused on ensuring an AI system's goals and behaviors are compatible with human values and ethical principles. Bias mitigation is a foundational technical requirement for achieving value alignment.

• Scope: Encompasses broader philosophical goals beyond immediate statistical fairness, including long-term safety and beneficence.
• Technical Paths: Implemented via techniques like Reinforcement Learning from Human Feedback (RLHF), Constitutional AI, and Direct Preference Optimization (DPO).
• Outcome: A well-aligned model inherently avoids discriminatory outputs and harmful biases as a consequence of its training to be helpful, honest, and harmless.

Preference modeling is the machine learning task of training a model to predict human or AI preferences between different outputs. It is central to data-driven bias mitigation through alignment.

• Function: The preference model, or reward model, learns nuanced judgments about output quality, safety, and fairness from labeled data.
• Process: Used in RLHF and RLAIF to provide training signals that steer the base model away from biased or harmful generations.
• Data Quality: The effectiveness of bias mitigation hinges on the quality and representativeness of the preference data, which must itself be scrutinized for annotator bias.

Direct Preference Optimization (DPO) is an efficient algorithm for aligning language models with human preferences using a dataset of preferred and dispreferred responses. It provides a stable method for bias mitigation through alignment.

• Mechanism: DPO reformulates the RLHF pipeline, directly optimizing the policy language model without training a separate reward model, improving stability.
• Efficiency: Reduces computational complexity and hyperparameter tuning, making advanced alignment and bias mitigation more accessible.
• Application: By training on datasets where biased outputs are marked as dispreferred, DPO directly learns to generate fairer, less harmful text.