Fairness Metric

Demographic Parity, also known as statistical parity, requires the overall rate of positive predictions to be equal across groups. It is defined as P(Ŷ=1 | A=a) = P(Ŷ=1 | A=b) for all groups a, b, where Ŷ is the model's prediction and A is the protected attribute.

• Use Case: Screening systems where the selection rate must be balanced, such as initial resume filtering.
• Limitation: Ignores differences in qualification rates between groups, which can lead to qualified individuals in one group being rejected to meet the quota.

Equal Opportunity requires the true positive rate (recall) to be equal across groups. It is defined as P(Ŷ=1 | Y=1, A=a) = P(Ŷ=1 | Y=1, A=b), where Y is the true label. This ensures qualified individuals from each group have an equal chance of receiving the beneficial outcome.

• Use Case: Credit lending or hiring, where the goal is to correctly identify all qualified applicants regardless of group membership.
• Focus: Mitigates disparate impact on qualified members of a protected class.

Equalized Odds is a stricter criterion that requires both the true positive rate and the false positive rate to be equal across groups: P(Ŷ=1 | Y=y, A=a) = P(Ŷ=1 | Y=y, A=b) for y ∈ {0,1}.

• Implication: The model must have equal error rates (both Type I and Type II) for all groups.
• Use Case: High-stakes decisions like criminal justice risk assessments, where both falsely flagging low-risk individuals and missing high-risk individuals must be balanced across demographics.
• Trade-off: Often requires a more significant accuracy trade-off than Equal Opportunity alone.

Predictive Parity, also known as outcome test, requires the positive predictive value (precision) to be equal across groups. It is defined as P(Y=1 | Ŷ=1, A=a) = P(Y=1 | Ŷ=1, A=b).

• Interpretation: When the model predicts a positive outcome, that prediction should be equally reliable (likely to be correct) for all groups.
• Use Case: Medical diagnostics, where the confidence in a positive test result must be consistent across patient demographics.
• Limitation: It is mathematically impossible to satisfy both Predictive Parity and Equal Opportunity simultaneously if the base rates (prevalence of Y=1) differ between groups, a result known as Bayesian impossibility.

Counterfactual Fairness is a causal, individual-level fairness notion. A model is counterfactually fair if, for any individual, its prediction would not change in a hypothetical world where that individual's protected attribute (e.g., race) was different, while keeping all other relevant, non-discriminatory circumstances the same.

• Foundation: Relies on building a causal model of the data-generating process to estimate these counterfactuals.
• Use Case: Complex decision systems where attributes are causally related, such as loan applications where zip code (a proxy for race) influences credit history.
• Challenge: Requires strong assumptions and a correctly specified causal graph, which is often difficult to obtain.

Treatment Equality focuses on balancing the types of errors made across groups by requiring the ratio of false negatives to false positives to be equal. It is defined as (FN_a / FP_a) = (FN_b / FP_b), where FN and FP are counts of false negatives and false positives per group.

• Focus: Directly addresses the distribution of harms from incorrect predictions.

• Use Case: Resource allocation systems where the cost of a false negative (denying a needed service) and a false positive (providing an unneeded service) must be balanced equitably across communities.

• Consideration: Unlike rate-based metrics, it is a count-based metric, making it sensitive to group sizes in the evaluation set.

Before model training, fairness metrics diagnose representation bias and historical bias in the training dataset.

• Demographic Parity is calculated on the training labels to check if positive outcomes (e.g., 'approved') are equally distributed across groups, revealing skewed historical decisions.
• Subgroup Analysis of base rates (prevalence of the target variable) highlights underrepresentation. A significant disparity warns of poor model generalizability for that group.
• This stage answers: Does my training data reflect equitable realities or perpetuate past inequities?

During training, fairness metrics are formalized as fairness constraints to directly shape the model's objective function.

• Equalized Odds or Equal Opportunity can be added as a Lagrangian constraint, forcing the optimizer to balance accuracy with group-wise performance parity.
• Adversarial debiasing uses a secondary model that tries to predict the protected attribute from the primary model's embeddings; the primary model is penalized for creating representations that leak group information.
• This in-processing application ensures fairness is baked into the model's parameters, not just patched later.

This is the core application: quantitatively comparing model performance across subgroups defined by protected attributes.

• Aggregate metrics (overall accuracy of 92%) are decomposed via subgroup analysis to reveal disparities (e.g., accuracy of 95% for Group A vs. 82% for Group B).
• Equal Opportunity (equal true positive rates) is critical for high-stakes applications like hiring or lending, ensuring qualified candidates from all groups have the same chance of being correctly identified.
• Equalized Odds (equal TPR and FPR) is a stricter test used when both false positives and false negatives carry significant cost.

For a fixed, already-trained model, fairness metrics guide adjustments to decision thresholds per subgroup to meet a target fairness criterion.

• To achieve Demographic Parity, different classification thresholds are applied to each group's score distribution to equalize the rate of positive predictions.
• To achieve Equal Opportunity, thresholds are tuned to equalize the true positive rate across groups.
• This is a pragmatic deployment-stage intervention but has limitations: it requires knowledge of the protected attribute at inference and may reduce overall accuracy.

Fairness metrics are tracked continuously on live model predictions to detect bias drift.

• Demographic Parity of predictions is monitored over time. A drift indicates the model is becoming disproportionately favorable/unfavorable to a group.
• Subgroup performance metrics (precision, recall) are tracked via A/B testing frameworks. A widening gap signals degrading fairness due to shifting input data distributions.
• This operationalizes fairness as a continuous Service Level Objective (SLO), triggering alerts and model retraining when fairness thresholds are breached.

Fairness metrics provide the empirical evidence required for algorithmic impact assessments, model cards, and regulatory compliance.

• A comprehensive bias audit report includes a table of key fairness metrics (Demographic Parity, Equal Opportunity, Predictive Equality) across all relevant protected classes and intersections.
• Model Cards publicly document these metrics, along with intended use cases and known fairness limitations, enabling informed third-party use.
• Metrics like Disparate Impact Ratio (a form of demographic parity) are directly referenced in legal frameworks (e.g., the U.S. Equal Employment Opportunity guidelines).

Algorithmic fairness is the study and application of principles and techniques to ensure that automated decision-making systems do not create or perpetuate unjust or discriminatory outcomes against individuals or groups based on sensitive attributes. It is the overarching ethical goal that fairness metrics are designed to quantify and enforce.

• Core Objective: To translate ethical principles into verifiable, mathematical criteria.
• Key Challenge: Different mathematical definitions of fairness (e.g., demographic parity, equal opportunity) are often mutually exclusive, requiring trade-offs based on context.

Disparate impact is a form of algorithmic bias that occurs when a model's outputs, while facially neutral, have a disproportionately adverse effect on members of a protected group, even without intentional discrimination. It is a central legal concept in U.S. anti-discrimination law (e.g., the 80% rule).

• Quantification: Often measured by comparing selection rates (e.g., loan approval rates) across groups.
• Example: A resume screening model that approves 60% of male applicants but only 30% of female applicants exhibits disparate impact, even if gender was not an explicit input feature.

A bias audit is a systematic, documented evaluation of an AI system to detect, measure, and report on potential discriminatory biases in its data, model, or outputs against defined protected groups. It is the practical process that employs fairness metrics.

• Key Components: Includes defining protected attributes, selecting relevant fairness metrics, conducting subgroup analysis, and documenting findings.
• Regulatory Driver: Mandated by laws like New York City's Local Law 144 for automated employment decision tools.

Subgroup analysis is the practice of evaluating a model's performance metrics separately for distinct demographic slices. Intersectional analysis extends this to subgroups defined by multiple protected attributes (e.g., Black women).

• Purpose: To identify performance disparities masked by aggregate metrics. A model with 95% overall accuracy might have 99% accuracy for Group A and 70% for Group B.
• Critical Insight: Bias is often most severe at the intersections of identities, which single-attribute analysis can miss.

Bias mitigation refers to technical interventions applied to reduce unfair discrimination. Techniques are categorized by when they are applied in the ML lifecycle:

• Pre-processing: Modifies the training data (e.g., reweighting samples, transforming features).
• In-processing: Adds fairness constraints or adversarial components to the training objective.
• Post-processing: Adjusts a trained model's predictions or decision thresholds per group.

Each approach has trade-offs between fairness, accuracy, and implementation complexity.

A protected attribute is a legally or ethically sensitive characteristic (e.g., race, gender, age) that cannot be used for discriminatory treatment. A proxy variable is a correlated feature (e.g., zip code, shopping patterns, name analysis) that can inadvertently allow a model to discriminate based on the protected attribute, even when it is explicitly removed from the data.

• Core Risk: Simply omitting a protected attribute is insufficient for fairness, as models can easily learn proxies from other features.
• Mitigation: Requires techniques like adversarial debiasing to learn representations that are invariant to the protected attribute.