Algorithmic Bias

Algorithmic bias is defined by its systematic nature—it produces repeatable, non-random errors that consistently skew in a particular direction. Unlike statistical variance or noise, which can be reduced with more data, systematic bias persists regardless of sample size. This distinction is critical: a model with high variance may give different wrong answers each time, but a biased model gives the same wrong answer for the same group, creating entrenched patterns of unfairness that compound over time.

Bias can enter at any stage of the machine learning lifecycle, not just during model training. Key entry points include:

• Data Collection: Sampling bias, underrepresentation, or historical prejudices encoded in labels
• Feature Engineering: Proxy variables that correlate with protected attributes, such as zip code standing in for race
• Model Selection: Objective functions that optimize for aggregate accuracy while ignoring subgroup performance
• Deployment Context: Feedback loops where biased predictions shape future training data, amplifying initial disparities Identifying the specific origin point is essential for selecting the appropriate mitigation technique.

One of the most insidious characteristics of algorithmic bias is its ability to operate through proxy variables—features that are not explicitly protected attributes but are statistically correlated with them. A model denied direct access to race may still discriminate by learning to weight features like geographic location, surname analysis, or purchasing patterns that encode racial information. This makes 'fairness through unawareness'—simply removing protected attributes—a demonstrably ineffective strategy, as modern ML models excel at reconstructing sensitive information from seemingly innocuous data points.

Bias does not operate along a single axis. Intersectional bias occurs when individuals belonging to multiple marginalized groups experience compounded disadvantage that exceeds the sum of individual biases. For example, a hiring model might perform adequately for women overall and for racial minorities overall, but fail dramatically for women of a specific racial minority. Standard fairness metrics that evaluate only one protected attribute at a time—a practice called single-axis analysis—systematically miss these intersectional failures, creating a false sense of fairness while the most vulnerable subgroups remain unprotected.

Deployed models do not operate in a vacuum—their predictions shape the world they measure. A feedback loop occurs when a biased model's outputs influence future training data, creating a self-reinforcing cycle:

• A predictive policing model sends more officers to a historically over-policed neighborhood
• More arrests are recorded in that area, appearing to 'confirm' the model's prediction
• The next training cycle learns from this distorted data, further concentrating enforcement This dynamic means that even small initial biases can amplify exponentially over time, transforming minor disparities into severe, entrenched inequities.

A defining characteristic of algorithmic bias is the accuracy-fairness trade-off—the observed tension where enforcing fairness constraints can reduce overall model accuracy. This occurs because real-world data often contains genuine statistical differences between groups that are themselves products of historical injustice. A model optimized purely for accuracy will exploit these patterns, while a fairness-constrained model must deliberately sacrifice some predictive power. The trade-off is not absolute—techniques like multicalibration and causal fairness aim to minimize it—but acknowledging its existence is essential for honest discussions about what 'fair' means in a mathematical context.