Predictive Parity

Predictive parity is satisfied when P(Y=1 | Ŷ=1, A=a) = P(Y=1 | Ŷ=1, A=b) for all groups a and b. In simpler terms, if the model predicts a positive outcome, the probability that this outcome is truly positive must be the same for every demographic group. This metric focuses exclusively on the precision of the positive class, making it a sufficiency criterion—the prediction score is sufficient to determine the outcome, and group membership adds no further information.

The metric is formally defined as the equality of the conditional probability of the true label given a positive prediction. For a binary classifier with a risk score R and threshold t, it requires:

• P(Y=1 | R > t, A=a) = P(Y=1 | R > t, A=b)

This is equivalent to requiring that the precision metric be equal across groups. A key implication is that a single, global decision threshold can be used for all groups while satisfying this condition, unlike other fairness metrics that may require group-specific thresholds.

Predictive parity is a form of group-wise calibration for the positive class. A model is well-calibrated if, for all instances assigned a probability score of s, the fraction of true positives is exactly s. Predictive parity specifically requires this calibration to hold within each group for the positive prediction outcome. If a model is well-calibrated for all groups, it automatically satisfies predictive parity. However, predictive parity alone does not guarantee full calibration across all score ranges.

A fundamental result in algorithmic fairness is the impossibility theorem: except in trivial cases, predictive parity cannot be simultaneously satisfied with equalized odds (equality of both true positive and false positive rates) and demographic parity when base rates differ between groups. If the prevalence of a positive outcome (Y=1) is different across groups, a model cannot have both equal precision and equal error rates. This forces practitioners to choose which fairness definition aligns with their ethical and legal context.

Predictive parity is the preferred fairness metric in scenarios where a positive prediction triggers a high-cost intervention or a valuable opportunity. Common applications include:

• Loan approval: Ensuring that among all applicants approved for a loan, the rate of successful repayment is the same across all demographic groups.
• University admissions: Guaranteeing that admitted students have an equal probability of graduating, regardless of background.
• Pre-trial release: Ensuring that among those predicted to be low-risk, the actual recidivism rate is equal across groups.

In these cases, the cost of a false positive is high, making precision the paramount concern.

A critical limitation is that predictive parity can mask systemic bias by perpetuating historical inequalities. If a disadvantaged group has been historically denied loans, the subset of that group that does receive loans may be exceptionally creditworthy, leading to a high PPV. A model satisfying predictive parity would then only approve the most exceptional candidates from that group, reinforcing the status quo. It also ignores false negatives, meaning it does not measure how many qualified individuals from a disadvantaged group are wrongly denied an opportunity.

A separation-based fairness metric requiring a classifier to have equal true positive rates and equal false positive rates across different sensitive groups. Unlike predictive parity, which conditions on the prediction, equalized odds conditions on the true outcome. This ensures that the model's errors are distributed equally, but it can conflict with calibration when base rates differ across groups.

An independence-based fairness criterion requiring a model's positive prediction rate to be equal across all groups defined by a protected attribute. Also known as statistical parity, this metric focuses solely on the distribution of predictions, ignoring the true outcome. It is often legally motivated but can permit 'lazy' solutions that select unqualified individuals from disadvantaged groups to meet the quota.

A relaxed version of equalized odds that constrains only the true positive rate to be equal across groups. This ensures that among individuals who would succeed, the same proportion is correctly identified regardless of group membership. It does not constrain false positive rates, making it a weaker but more achievable fairness standard than full equalized odds.

A sufficiency-based criterion requiring that for any predicted risk score, the probability of actually belonging to the positive class is the same across groups. Calibration and predictive parity are closely related: if a classifier is well-calibrated for all groups, predictive parity is automatically satisfied. However, calibration and equalized odds are provably incompatible when base rates differ.

A causal definition of fairness where a prediction for an individual is considered fair if it remains the same in the actual world and a counterfactual world where the individual belonged to a different demographic group. This approach requires a structural causal model and goes beyond purely statistical metrics like predictive parity by explicitly modeling the causal pathways of discrimination.

A fairness principle requiring that similar individuals receive similar predictions, formalized by a distance metric constraint on the input and output spaces. Unlike predictive parity, which is a group-level metric, individual fairness operates at the granularity of specific pairs of individuals. This approach avoids the statistical averaging that can mask unfairness against intersectional subgroups.