Equalized Odds

Equalized Odds requires simultaneous parity in two key metrics: the True Positive Rate (TPR) and the False Positive Rate (FPR). A model satisfies this criterion if, for any two protected groups A and B, the probability of a positive prediction given a positive instance is equal, and the probability of a positive prediction given a negative instance is also equal. This dual constraint prevents a model from masking high false positive rates in one group with high true positive rates in another.

Formally, for a predictor Ŷ and a protected attribute A, Equalized Odds holds if: P(Ŷ=1 | Y=y, A=a) = P(Ŷ=1 | Y=y, A=b) for all y ∈ {0,1} and all groups a, b. This means the predictor Ŷ is conditionally independent of the sensitive attribute A given the true outcome Y. Unlike Demographic Parity, this definition explicitly uses the ground truth label, making it a separation-based metric that aligns model errors with actual outcomes rather than just prediction distributions.

Equalized Odds occupies a specific position in the fairness taxonomy:

• Stronger than Demographic Parity: It conditions on the true label, making it outcome-aware.
• Weaker than Calibration by Group: A perfectly calibrated system automatically satisfies Equalized Odds, but the reverse is not true.
• Incompatible with Predictive Parity when base rates differ across groups. A model cannot simultaneously satisfy Equalized Odds and equal Positive Predictive Value unless prevalence is identical across all groups.

The COMPAS recidivism prediction tool became a landmark case study. An investigation by ProPublica found that the algorithm had similar AUC and calibration across racial groups but violated Equalized Odds. Specifically, Black defendants who did not reoffend were nearly twice as likely to be classified as high-risk compared to White defendants (higher FPR), while White defendants who did reoffend were more likely to be misclassified as low-risk (lower TPR). This demonstrated that calibration alone is insufficient for fairness.

Equalized Odds has notable limitations:

• Relies on Ground Truth: It requires accurate, unbiased labels Y, which are often unavailable in historically biased domains like hiring or policing.
• Ignores Legitimate Risk Factors: By forcing equal error rates, it may prevent a model from using genuinely predictive features that correlate with group membership, potentially reducing overall accuracy.
• Does Not Guarantee Individual Fairness: Two similar individuals from different groups could receive different outcomes if base rates differ, as the criterion operates at the group level.

A group fairness metric requiring a model's positive prediction rate to be statistically identical across all protected groups. This ensures the decision is independent of sensitive attributes like race or gender. A common legal benchmark is the 80% rule, where the selection rate for any group must be at least 80% of the group with the highest rate. While intuitive, it can be problematic when base rates of a legitimate qualification differ between groups, potentially forcing the model to make incorrect predictions to satisfy the constraint.

A causal definition of fairness where a decision is considered fair if it would remain the same in a counterfactual world where an individual's sensitive attributes were different. This requires a structural causal model to reason about interventions. For example, a loan application would be counterfactually fair if the decision for a female applicant would have been identical had she been male, holding all other causally dependent features constant. It is a highly robust, individual-level definition but is difficult to implement without a correct causal graph.

A fairness criterion ensuring that a model's predicted probabilities accurately reflect the true likelihood of an outcome for every distinct demographic group. If a model predicts a 10% recidivism risk for both Black and white defendants, the proportion of those who actually re-offend must be 10% in both groups. This prevents over- or under-estimation of risk for specific populations, which is critical in high-stakes domains like healthcare and criminal justice. It is also known as test-fairness or sufficiency.

A legal doctrine and quantitative measure of discrimination where a facially neutral policy disproportionately harms members of a protected group. It is often assessed using the adverse impact ratio, calculated as the selection rate of a protected group divided by the selection rate of the majority group. A ratio below 0.8 (the 80% rule) is a prima facie indicator of discrimination under U.S. employment law. Unlike other metrics, it focuses on the outcome of a process, not its intent.

The inherent tension in model optimization where enforcing strict fairness constraints often results in a measurable reduction in the system's overall predictive accuracy or business utility. This trade-off is not always linear; a small relaxation in accuracy can sometimes yield a large gain in fairness. The optimal operating point is an ethical and business decision, often visualized on a Pareto frontier that plots fairness metrics against model performance to guide stakeholder selection.

The ability to provide a clear, actionable path for individuals to reverse an unfavorable algorithmic decision. It identifies the minimal set of actionable features a person must change to achieve a desired outcome. For instance, a loan denial system with recourse might tell an applicant: 'Increase your credit score by 15 points and reduce your debt-to-income ratio by 2% to qualify.' This concept shifts the focus from mere explanation to empowerment and agency, making automated decisions contestable.