Bias Correction

This foundational technique addresses bias at its source by modifying the training dataset before model training begins. The goal is to create a more balanced data distribution.

• Reweighting: Assigning higher importance (weights) to underrepresented examples during the loss calculation.
• Resampling: Oversampling minority groups or undersampling majority groups to balance class distributions.
• Data Augmentation: Generating synthetic examples for underrepresented concepts or demographics.
• Adversarial Filtering: Using a classifier to identify and remove stereotypical associations from text corpora before training.

Example: In a hiring model, resampling the training data to ensure equal representation of candidates across different demographic groups.

These methods modify the learning objective itself by adding constraints or penalties to the loss function, forcing the model to learn fairer representations during training.

• Regularization Terms: Adding a penalty to the loss function that measures a fairness metric (e.g., demographic parity, equalized odds). The model is optimized for accuracy and fairness.
• Adversarial Debiasing: Training a main model alongside an adversarial network that tries to predict a protected attribute (e.g., gender, race) from the main model's internal representations. The main model is trained to prevent the adversary from succeeding, thus learning to discard bias-related information.
• Contrastive Learning: Using loss functions that pull similar examples together and push dissimilar ones apart in the embedding space, which can be designed to reduce correlation with sensitive attributes.

These are applied to a model's outputs after it has generated them, but before they are delivered to the user. They act as a final safety filter.

• Calibration: Adjusting prediction probabilities (scores) for different subgroups to ensure they reflect true likelihoods equally. A score of 0.8 should mean an 80% chance of being correct for all groups.
• Threshold Optimization: Setting different classification thresholds for different demographic groups to achieve fairness metrics like equal opportunity.
• Output Rewriting/Reranking: Using a secondary model or rule-based system to detect biased phrasing in generated text and rewrite it, or to rerank multiple candidate outputs based on fairness scores.

Limitation: Treats the symptom, not the cause, and can be computationally expensive at scale.

A targeted fine-tuning approach where the model is explicitly trained on examples designed to teach it to avoid specific biases.

• Counterfactual Data Augmentation (CDA): Creating training examples by swapping sensitive attributes in text. For instance, taking a sentence "The nurse prepared the medicine" and creating a counterfactual "The doctor prepared the medicine." The model is fine-tuned on both, learning that the core task is independent of the swapped attribute.
• Stereotype Negation: Fine-tuning the model on explicit statements that negate common stereotypes (e.g., "Women are not less skilled at mathematics.").
• Refusal Training for Biased Prompts: Extending refusal training to include prompts that ask for biased or stereotypical content, teaching the model to reject such requests.

This advanced technique uses causal graphs to formally model and identify the pathways through which bias affects model predictions. It moves beyond correlation to understand causation.

• Causal Graphs: Diagrams that represent variables (e.g., Zip Code, Socioeconomic Status, Loan Approval) and the causal relationships between them.
• Intervention: The statistical 'do-operator' allows engineers to simulate what would happen if a sensitive attribute were changed while holding other factors constant, isolating its direct effect.
• Path-Specific Analysis: Identifying which causal pathways from a sensitive attribute to an outcome are direct (illegitimate bias) versus indirect (mediated by a permissible factor, like qualifications).

This framework provides a rigorous, interpretable method for diagnosing bias and designing precise correction techniques.

Bias correction is not a one-time task but requires continuous monitoring and adaptation in production systems, integrated into Safety Fine-Tuning Loops.

• Disaggregated Evaluation: Continuously measuring model performance (accuracy, F1 score) and output metrics across defined demographic subgroups, not just on aggregate.
• Bias Drift Detection: Using statistical tests to monitor for concept drift specific to fairness metrics, alerting when a model's behavior becomes significantly more biased.
• Human-in-the-Loop Audits: Regularly sampling production outputs for human review to identify novel or emergent biases not captured by automated metrics.
• Feedback Integration: Using audit results and user feedback on biased outputs to create new data for the retraining pipeline, closing the loop for continuous correction.

Safety alignment is the overarching goal of ensuring an AI model's objectives and outputs are consistent with human values, ethical principles, and safety constraints. It is the super-category that encompasses bias correction.

• Goal: To create models that are helpful, harmless, and honest.
• Methods: Includes constitutional AI, RLHF, and adversarial training.
• Scope: Addresses a wider range of risks than just bias, including misuse, deception, and power-seeking behaviors.

RLHF is a primary technique for safety alignment where a model is fine-tuned using preferences expressed by human raters. A reward model is trained to predict which outputs humans prefer, and this reward signal guides the main model's training via reinforcement learning.

• Process: 1) Collect human comparisons of model outputs. 2) Train a reward model on these preferences. 3) Use Proximal Policy Optimization (PPO) to fine-tune the main model to maximize the predicted reward.
• Role in Bias: Can be used to directly optimize for less biased outputs if the preference data explicitly ranks responses for fairness.

DPO is a stable and efficient alternative to RLHF for aligning models to preferences. It directly optimizes a policy using a loss function derived from preference data, eliminating the need for a separate reward model and the complex PPO reinforcement learning loop.

• Advantage: Simpler, more stable, and computationally cheaper than RLHF.
• Mechanism: Treats the language model itself as the implicit reward function, optimizing it to increase the log-likelihood of preferred responses and decrease that of dispreferred ones.
• Application: Highly effective for bias correction when provided with preference pairs where the preferred response demonstrates less bias.

Adversarial fine-tuning is a training process that exposes a model to deliberately crafted harmful or biased prompts (adversarial examples) during fine-tuning to improve its robustness and safety. The model learns to refuse or correctly handle these challenging inputs.

• Purpose: To proactively 'inoculate' the model against potential attacks or failure modes.
• Data Source: Often uses outputs from red teaming exercises, where specialists try to generate harmful outputs.
• Relation to Bias: A key method for correcting societal bias by fine-tuning the model on adversarial prompts designed to elicit stereotypical or unfair outputs.

A safety dataset is a curated collection of prompts and responses used to train or evaluate an AI model's adherence to safety and fairness guidelines. For bias correction, these datasets are critical.

• Content: Includes examples of harmful queries, biased statements, model refusals, and benign interactions.
• Types:
  • Bias Benchmark for QA (BBQ): Measures social biases in question-answering.
  • Toxicity Datasets: For hate speech and harassment mitigation.
  • Adversarial NLI: Tests for stereotypical reasoning.
• Usage: Used to fine-tune models, train reward models for RLHF, and serve as evaluation benchmarks.

Red teaming is a security practice where a dedicated team systematically attempts to generate adversarial inputs to expose vulnerabilities, harmful behaviors, or biases in an AI model. It is a proactive discovery phase that informs correction techniques.

• Process: Experts craft 'jailbreak' prompts, role-play malicious users, and stress-test the model's boundaries.
• Output: The generated failure cases (prompts and harmful outputs) become valuable data for adversarial fine-tuning and for expanding safety datasets.
• Goal: To find and fix problems, including biased reasoning, before they are encountered in production.