Equalized Odds

Equalized Odds simultaneously enforces two statistical parity conditions: the True Positive Rate (TPR) and the False Positive Rate (FPR) must be identical across all sensitive groups. This is stricter than Equal Opportunity, which only constrains the TPR. The dual constraint ensures that a classifier is equally accurate for all groups in both its positive and negative predictions, preventing a model from compensating for a high FPR in one group with a high TPR in another.

For a binary predictor Ŷ, true outcome Y, and protected attribute A, Equalized Odds is satisfied if:

• P(Ŷ=1 | Y=1, A=a) = P(Ŷ=1 | Y=1, A=b) for all groups a, b
• P(Ŷ=1 | Y=0, A=a) = P(Ŷ=1 | Y=0, A=b) for all groups a, b

This definition is grounded in the separation criterion from the fairness taxonomy by Hardt, Price, and Srebro (2016). It requires the score distribution to be independent of the protected attribute, conditioned on the true label.

Equalized Odds occupies a specific position in the fairness landscape:

• vs. Demographic Parity: Demographic Parity requires equal positive prediction rates unconditionally, which can force a classifier to select unqualified individuals from disadvantaged groups. Equalized Odds avoids this by conditioning on the true outcome.
• vs. Equal Opportunity: Equal Opportunity is a relaxation that only constrains the TPR, making it a subset of Equalized Odds.
• vs. Predictive Parity: Predictive Parity conditions on the prediction rather than the true label. A classifier cannot simultaneously satisfy Equalized Odds and Predictive Parity unless the base rates are equal across groups.

A fundamental impossibility result states that Equalized Odds and calibration cannot be simultaneously satisfied when base rates differ across groups. If a model is perfectly calibrated, meaning a score of 0.8 implies an 80% chance of success for all groups, then the TPR and FPR will necessarily differ if one group has a higher prevalence of positive outcomes. This trade-off forces practitioners to choose between individual fairness (calibration) and group error-rate parity (Equalized Odds).

Equalized Odds can be enforced through a post-processing algorithm that derives separate decision thresholds for each group from a learned score distribution. The Hardt method solves a constrained optimization problem: it finds group-specific thresholds that minimize a loss function while satisfying the TPR and FPR equality constraints. This approach is model-agnostic, requiring only access to the model's scores and the true labels on a validation set, making it suitable for auditing black-box classifiers.

When implementing Equalized Odds in production systems:

• Sample Size Sensitivity: Reliable estimation of TPR and FPR requires sufficient samples from each protected group, especially for rare outcomes.
• Multi-Class Extension: For multi-class problems, the constraint generalizes to requiring equal confusion matrix diagonals and off-diagonals across groups.
• Continuous Scores: For risk assessment tools, the metric is often applied by checking for equality of the ROC curves across groups, a condition known as ROC fairness.
• Intersectional Groups: Applying Equalized Odds to intersectional subgroups (e.g., race and gender combined) dramatically increases the number of constraints and the difficulty of satisfying them simultaneously.