Word Embedding Association Test (WEAT)

These are the primary concept pairs whose relative association is being tested. Each set contains words representing a specific social category.

• Example 1 (Gender): Target Set A: {man, male, boy, brother, he, him}; Target Set B: {woman, female, girl, sister, she, her}.
• Example 2 (Race): Target Set A: {European, American, White}; Target Set B: {African, Mexican, Black}. The test measures which attribute word sets are statistically closer to each target set in the embedding space.

These are the paired sets of words representing the attributes or stereotypes being measured. The core calculation determines which target set is more strongly associated with each attribute set.

• Classic Example: Attribute Set X (Career): {executive, management, professional, corporation, salary}; Attribute Set Y (Family): {home, parents, children, family, wedding}. The WEAT score quantifies if, for instance, male target words are systematically closer to career words and female target words are closer to family words within the model's geometry.

This is the standardized mean difference in association strengths, quantifying the magnitude of the detected bias. It is calculated as: d = (mean_similarity(A, X) - mean_similarity(A, Y)) - (mean_similarity(B, X) - mean_similarity(B, Y)) / (pooled_standard_deviation)

• A positive d indicates Target Set A is more associated with Attribute X than Y, relative to Set B.
• The value's magnitude (e.g., d = 1.5) indicates the strength of the effect, with common benchmarks (e.g., 0.2=small, 0.5=medium, 0.8=large) borrowed from psychology.

A non-parametric statistical test used to determine the significance of the observed effect size. It assesses the probability that the observed association difference occurred by random chance.

• Process: The labels of the target words are randomly shuffled thousands of times, and a null distribution of effect sizes is computed.
• The p-value is the proportion of permutations where the randomized effect size equals or exceeds the observed effect size.
• A low p-value (e.g., p < 0.05) provides evidence that the observed bias is statistically significant and not an artifact of random sampling.

The fundamental geometric operation used to measure association strength between individual words. For two word vectors u and v, cosine similarity is defined as: cosine_similarity(u, v) = (u · v) / (||u|| ||v||)

• It measures the cosine of the angle between vectors, ranging from -1 (opposite) to +1 (identical direction).
• In WEAT, the mean cosine similarity between all words in a target set and all words in an attribute set is computed. This reliance on vector direction makes the test sensitive to semantic relationships encoded by the embedding model.

WEAT is used to audit foundational models like BERT or GPT for learned stereotypes before deployment. It quantifies associations between:

• Target concepts (e.g., career, family)
• Attribute concepts (e.g., male, female names) By calculating the effect size (Cohen's d) and statistical significance (p-value), it provides a standardized report on gender, racial, or other social biases encoded in the embedding geometry. This is a critical first step in the model card creation process.

Researchers and engineers use WEAT as a quantitative benchmark to evaluate the efficacy of bias mitigation strategies. By applying WEAT before and after interventions like:

• Adversarial debiasing
• Counterfactual data augmentation
• Projection-based neutralization Teams can measure the reduction in association strength. A successful technique should show a statistically significant decrease in the WEAT effect size while preserving the model's utility on core tasks.

In production systems, WEAT can be integrated into continuous monitoring pipelines to detect bias drift. As new data fine-tunes a model or the underlying corpus statistics shift, previously mitigated associations can re-emerge. Regular WEAT evaluations on held-out concept sets act as a canary analysis for fairness, triggering alerts when effect sizes exceed predefined thresholds, ensuring ongoing compliance with algorithmic impact assessments.

While standard WEAT tests single associations, its methodology can be extended for intersectional analysis. This involves constructing target and attribute sets that represent compound identities (e.g., 'Black women' professionals vs. 'White men' professionals). This reveals compounded biases not visible in single-attribute tests, providing a more nuanced audit that aligns with modern fairness metric design for complex social realities.

WEAT is applied to the word embeddings trained on candidate datasets to audit for inherited stereotypes. This is crucial when curating or generating synthetic data for model training. By testing the embeddings derived from a new corpus, teams can assess its bias footprint before committing costly compute resources to full model training, implementing a pre-processing bias mitigation strategy at the data source.

The insights from WEAT directly inform the fairness constraint design in in-processing mitigation. By identifying which semantic directions in the embedding space are most problematic, engineers can design more targeted regularization terms or adversarial objectives. This moves bias mitigation from a black-box post-processing step to a principled component of the model training objective, supported by empirical measurement.

Algorithmic fairness is the engineering discipline focused on ensuring automated decision-making systems do not create or perpetuate unjust outcomes based on protected attributes like race or gender. It involves defining quantitative fairness goals, measuring disparities, and implementing technical mitigations.

• Core Challenge: Balancing model accuracy with equitable outcomes across groups.
• Key Principle: Different fairness definitions (e.g., demographic parity, equal opportunity) are often mathematically incompatible; the choice is a value-based engineering decision.

Bias in LLMs refers to the propensity of foundation models to generate outputs that reflect stereotypes present in their massive, web-scale training data. WEAT directly measures one manifestation of this: implicit association in word embeddings.

• Sources: Historical bias, representation bias, and linguistic correlations in pretraining corpora.
• Audit Methods: Beyond WEAT, audits include prompt-based stereotype scoring, toxicity analysis, and subgroup performance evaluation on tailored benchmarks.

A fairness metric is a quantitative measure used to assess equity in model predictions across demographic subgroups. WEAT produces a statistical measure of association (an effect size), which is a specific type of metric for representational bias.

• Group Fairness Metrics: Include demographic parity (equal selection rates), equal opportunity (equal true positive rates), and equalized odds (equal TPR & FPR).
• Individual Fairness Metrics: Assess if similar individuals receive similar predictions, related to the geometric distances WEAT examines.

Bias mitigation comprises technical interventions applied during the ML lifecycle to reduce unfair discrimination. Discovering bias via WEAT typically leads to applying one of three mitigation classes:

• Pre-processing: Alter training data (e.g., reweighting, swapping word embeddings) to remove bias before model training.
• In-processing: Add fairness constraints or adversarial debiasing to the training objective.
• Post-processing: Adjust model outputs or decision thresholds for different groups after training.

Adversarial debiasing is a powerful in-processing mitigation technique. A primary model is trained to perform a task (e.g., sentiment analysis) while an adversarial model tries to predict the protected attribute (e.g., gender) from the primary model's internal representations.

• Mechanism: The primary model learns to make its representations invariant to the protected attribute, thereby removing that signal for discrimination.
• Connection to WEAT: This technique directly attacks the geometric associations that WEAT measures, aiming to decorrelate embedding directions from sensitive concepts.

A Model Card is a documentation framework for transparent reporting of a model's performance, including fairness limitations. A systematic bias audit, using tools like WEAT, provides the empirical findings for this documentation.

• Audit Process: Involves defining protected groups, selecting appropriate metrics (like WEAT), conducting subgroup and intersectional analysis, and reporting disparities.
• Outcome: The audit results are documented in the Model Card to inform users of known biases, enabling informed deployment decisions and setting a baseline for monitoring bias drift.