Demographic Parity

Demographic Parity requires that the probability of a positive prediction is identical across all demographic groups. Formally, this is expressed as P(ŷ=1 | A=a) = P(ŷ=1 | A=b) for any two groups a and b. This enforces that the model's output is statistically independent of the sensitive attribute, meaning knowing a person's group membership provides no information about the likelihood of a favorable classification. This is the strictest form of group fairness and is often legally mandated in contexts like hiring.

In practice, perfect equality is rarely achievable, so compliance is often measured using the Disparate Impact Ratio. This is calculated by dividing the positive prediction rate of the unprivileged group by that of the privileged group. The U.S. Equal Employment Opportunity Commission's 80% rule is a common threshold: a ratio below 0.8 indicates a potential adverse impact. A ratio of 1.0 represents perfect demographic parity.

A critical weakness of Demographic Parity is that it permits lazy solutions. A model can satisfy the constraint by randomly granting positive outcomes to unqualified individuals in one group to match the rate of qualified individuals in another, or by arbitrarily denying qualified candidates. This is known as tokenism or randomized fairness. The metric ignores the ground truth label (Y), meaning it does not account for whether the decisions were actually correct or justified.

Demographic Parity becomes mathematically impossible to satisfy without causing harm when the base rates of the target variable differ significantly between groups. If the true rate of loan default is 10% for Group A and 20% for Group B, forcing equal acceptance rates would require either:

• Over-accepting risky applicants in Group B
• Under-accepting qualified applicants in Group A This inherent conflict with predictive accuracy is the central fairness-utility trade-off.

A common technique to achieve Demographic Parity is reweighing, a pre-processing method. It works by assigning weights to training examples based on the combination of their sensitive attribute and outcome label. For instance, a qualified applicant from an underrepresented group receives a higher weight, while an unqualified applicant from an overrepresented group receives a lower weight. The model is then trained on this weighted dataset, learning to make predictions that are statistically independent of the sensitive attribute without explicit in-processing constraints.

Demographic Parity can be enforced during model training using adversarial debiasing. This architecture pits a predictor network against an adversary network. The predictor tries to accurately classify the target variable, while the adversary tries to predict the sensitive attribute from the predictor's output. The predictor is trained to maximize accuracy while simultaneously minimizing the adversary's ability to detect the protected group, effectively learning a representation that achieves statistical independence from the sensitive attribute.

A stricter fairness criterion requiring equal true positive rates and false positive rates across all protected groups. Unlike demographic parity, equalized odds conditions on the true outcome, ensuring that errors are distributed evenly.

• True Positive Rate Equality: The model correctly identifies positive cases at the same rate for all groups
• False Positive Rate Equality: The model incorrectly flags negative cases at the same rate for all groups
• Trade-off: Often conflicts with demographic parity when base rates differ across groups

A legal doctrine and quantitative measure originating from U.S. employment law, often assessed using the 80% rule (four-fifths rule). A selection process exhibits disparate impact if the selection rate for a protected group is less than 80% of the rate for the group with the highest selection rate.

• Formula: (Selection Rate of Protected Group) / (Selection Rate of Reference Group) < 0.80
• Key Distinction: Disparate impact focuses on outcomes, not intent
• Application: Widely used in lending, hiring, and criminal justice algorithmic audits

A causal definition of fairness where a decision is fair if it would remain the same in a counterfactual world where an individual's sensitive attribute was different, while all other causally dependent features are adjusted accordingly.

• Causal Foundation: Requires a structural causal model to define the relationships between attributes
• Individual-Level: Assesses fairness at the granularity of a single person rather than group averages
• Strength: Captures the downstream effects of sensitive attributes on seemingly neutral features like zip code or education

The systematic application of algorithmic techniques to reduce unwanted systematic errors across three intervention points:

• Pre-processing: Transform training data to remove bias before model training (e.g., reweighting, fair representation learning)
• In-processing: Add fairness constraints directly into the model's objective function during training (e.g., adversarial debiasing, fairness-aware regularization)
• Post-processing: Adjust model outputs after prediction to satisfy fairness criteria (e.g., threshold adjustment, reject option classification)

Each approach presents a different point on the fairness-utility trade-off curve.

A fairness criterion ensuring that a model's predicted probabilities accurately reflect the true empirical likelihood of an outcome for every distinct demographic group. A well-calibrated model means that among all instances receiving a predicted score of 0.8, approximately 80% should belong to the positive class, regardless of group membership.

• Sufficiency: The outcome is independent of the sensitive attribute given the predicted score
• Conflict: Calibration is often incompatible with equalized odds when base rates differ across groups
• Use Case: Critical in risk assessment tools where probability estimates drive resource allocation decisions

The overarching discipline of designing machine learning systems that make impartial decisions, avoiding unjust bias against individuals or groups based on protected attributes such as race, gender, age, or disability status.

• Group Fairness: Statistical parity across demographic groups (e.g., demographic parity, equalized odds)
• Individual Fairness: Similar individuals should receive similar outcomes regardless of group membership
• Causal Fairness: Decisions should be invariant to sensitive attributes in a causal sense

Governance frameworks like the EU AI Act increasingly mandate algorithmic fairness assessments for high-risk systems.