Debiasing

This pre-training technique involves modifying the training dataset to reduce the prevalence of biased associations. It is a proactive, upstream intervention.

• Bias-Scoring & Removal: Training data is scored for stereotypical associations (e.g., using pointwise mutual information) and high-bias examples are filtered out.
• Counterfactual Data Augmentation: New, balanced examples are synthetically generated. For instance, if a dataset contains "The nurse prepared the syringe," a counterfactual "The doctor prepared the syringe" is added to balance gender associations.
• Real-World Limitation: Over-aggressive filtering can degrade linguistic quality and remove necessary context, a challenge known as the robustness-fairness trade-off.

This method adapts a pre-trained model using specialized datasets or loss functions to steer it away from biased representations.

• Adversarial Debiasing: An adversarial network is trained to predict a protected attribute (e.g., gender) from the model's internal representations. The main model is simultaneously trained to perform its core task while fooling the adversary, forcing it to learn representations that are invariant to the bias.
• Counterfactual Logit Adjustment: During fine-tuning, the loss function is modified to penalize predictions that align with stereotypical associations, directly optimizing for demographic parity.
• Constitutional AI & Self-Critique: The model is fine-tuned to critique and revise its own outputs according to fairness principles outlined in a constitution.

These techniques operate during the model's inference (generation) phase to correct outputs in real-time, without retraining.

• Controlled Generation via Prompting: System prompts explicitly instruct the model to avoid stereotypes (e.g., "You are a fair and unbiased assistant"). This is simple but can be circumvented.
• Vocabulary & Logit Suppression: The probability (logit) of generating next tokens that are part of identified biased phrases is dynamically reduced before sampling.
• Discriminatory Phrase Blocklisting: A post-processing filter scans and blocks or rewrites outputs containing a predefined list of harmful stereotypical phrases. This is a blunt but fast guardrail.

This class of techniques directly modifies the model's internal embedding space—where words and concepts are represented as vectors—to neutralize bias.

• Linear Subspace Projection: Bias is identified as a linear direction in the embedding space (e.g., the vector from 'man' to 'woman'). Biased associations (e.g., 'programmer' being closer to 'man') are then neutralized by subtracting their projection onto this bias direction.
• Bolukbasi Method: A seminal technique that identifies the gender subspace via words like {he, she, man, woman} and then debiases occupation words by making them equidistant from gender anchors.
• Limitation: This treats bias as a simple, linear phenomenon, which may not capture its full complexity in modern, high-dimensional models.

Robust debiasing requires rigorous measurement. Specialized benchmarks quantify different facets of model bias.

• StereoSet: Measures stereotypical bias via intrasentence context associations, providing a score for model preference for stereotypical vs. anti-stereotypical completions.
• CrowS-Pairs: A benchmark for social bias across nine categories (race, gender, religion, etc.) using minimal pair sentences that differ only by a protected attribute.
• BiasNLI: Uses natural language inference to test if a model systematically finds stereotypical premises more entailed than anti-stereotypical ones.
• Holistic Evaluation: Effective debiasing uses a suite of benchmarks, as improving performance on one can sometimes degrade another (bias transfer).

These are structural modifications to the neural network itself, designed to compartmentalize or isolate biased knowledge.

• Bottleneck Adapters: Small, trainable modules are inserted into a frozen base model. Debiasing is performed by training only these adapters on balanced data, preventing catastrophic forgetting of general knowledge.
• Modular Networks: The model architecture is split into separate, gated components for factual knowledge and social reasoning. The goal is to allow selective inhibition of the "bias" module during sensitive tasks.
• Knowledge Localization & Editing: Techniques like ROME aim to precisely locate and edit specific factual (or biased) associations within the model's weights, moving towards surgical debiasing.