Protected Attribute

Protected attributes are often codified in anti-discrimination law. Common examples include:

• Race, Color, National Origin (U.S. Civil Rights Act)
• Sex/Gender (Title VII, EU Gender Directive)
• Religion or Creed
• Age (Age Discrimination in Employment Act)
• Disability (Americans with Disabilities Act)

The specific list varies by jurisdiction (e.g., the EU AI Act, U.S. state laws), making compliance a multi-regional challenge. Legal definitions establish the baseline for what constitutes a protected class in algorithmic audits.

These attributes are typically intrinsic, inherent, or fundamental to personal identity and are not chosen or easily changed. This characteristic is why they are legally protected from being used for discriminatory treatment.

Core Examples:

• Biological sex
• National origin
• Race
• Certain disabilities

Contextual Examples:

• Age (changes but is not a choice)
• Religion (may be chosen but is considered fundamental)
• Pregnancy status

Systems should not penalize individuals based on these innate or core characteristics.

A critical challenge in bias auditing is that protected attributes can be strongly correlated with other, seemingly neutral data features. Even if a protected attribute (e.g., race) is removed from the training data, a model can infer it and discriminate via these proxy variables.

Common Proxies:

• Zip/Postal Code: Highly correlated with race and socioeconomic status.
• Shopping Patterns: Can correlate with gender or age.
• Language Use: May correlate with national origin.
• Educational Institution: Can correlate with race and socioeconomic background.

Effective bias mitigation requires identifying and controlling for these proxies.

Protected attributes define the subgroups for fairness evaluation. Audits require slicing model performance metrics (accuracy, FPR, TPR) by these attributes to detect disparate impact.

Intersectional analysis takes this further by examining subgroups at the intersection of multiple attributes (e.g., Black women over 50). Bias is often compounded at these intersections, a fact masked by analyzing attributes in isolation.

Key Practice: Aggregate metrics like 95% overall accuracy are insufficient. Performance must be validated across all legally relevant subgroups to ensure equitable treatment.

Every quantitative fairness metric is defined in relation to protected attributes (A) and model predictions (Y). These metrics mathematically formalize non-discrimination.

Common Definitions:

• Demographic Parity: P(Y=1 | A=Group1) = P(Y=1 | A=Group2)
• Equal Opportunity: P(Y=1 | A=Group1, Actually Qualified) = P(Y=1 | A=Group2, Actually Qualified)
• Equalized Odds: Requires both equal opportunity and equal false positive rates across groups.

Choosing the appropriate metric depends on the context and which potential harm (allocation vs. quality-of-service) the system must avoid.

The set of protected attributes is not static. It evolves with societal norms, legal precedents, and cultural understanding.

Emerging Considerations:

• Sexual Orientation & Gender Identity: Explicitly protected in an increasing number of jurisdictions.
• Genetic Information: Protected under laws like GINA in the U.S.
• Veteran Status: Protected in employment contexts.
• Family Responsibilities: A protected ground in some regions.

Furthermore, the operational definition of an attribute (e.g., how 'gender' is categorized in a dataset) is itself a critical ethical and engineering decision that can introduce bias if poorly constructed.

A legal and technical concept describing a facially neutral model or policy that produces disproportionately adverse outcomes for members of a protected group, regardless of intent.

• Key Mechanism: Measured by comparing outcome rates (e.g., denial rates) across groups. A common threshold is the "80% rule" (or four-fifths rule), where a selection rate for a protected group less than 80% of the rate for the favored group may indicate disparate impact.
• Example: A resume screening model that rejects 2% of applicants from Group A but 10% from Group B, where the only differentiating factor is a proxy for a protected attribute like educational institution reputation.

Occurs when a model explicitly uses a protected attribute as a direct input feature to make different decisions for different groups, constituting direct discrimination.

• Contrast with Disparate Impact: Disparate treatment is about explicit use, while disparate impact is about unequal outcomes from seemingly neutral rules.
• Technical Implementation: This is often prevented by sensitive feature scrubbing—removing attributes like race or gender from training data. However, proxy variables can reintroduce this effect indirectly.
• Example: A credit scoring algorithm that has separate, explicit decision thresholds for "female" and "male" applicants.

A non-protected feature that is highly correlated with a protected attribute, allowing a model to effectively discriminate based on that attribute even when it is omitted from the dataset.

• Common Proxies: ZIP/postal code (correlates with race/income), shopping patterns, university name, or even linguistic patterns in text.
• Detection Challenge: Identifying proxies requires causal analysis and correlation testing. Techniques like residual analysis can uncover if a model's predictions still correlate with a protected attribute after it's removed.
• Mitigation: Methods include feature transformation to decorrelate proxies from outcomes or using adversarial debiasing to learn representations invariant to the protected attribute.

The evaluation of model performance and fairness metrics across subgroups defined by the combination of multiple protected attributes (e.g., race and gender, age and disability).

• Purpose: Recognizes that bias is often compounded, not additive. A model may perform fairly for "women" and fairly for "Black individuals" but fail catastrophically for "Black women."
• Technical Method: Involves slicing evaluation data by Cartesian products of protected attributes and computing metrics per intersectional slice. This can expose severe disparities masked by one-dimensional analysis.
• Scalability Challenge: The number of subgroups grows exponentially, requiring significant data and careful statistical handling to avoid small sample sizes.

A systematic, documented procedure to detect, measure, and report on discriminatory biases in an AI system's data, model, or outputs against legally or ethically defined protected groups.

• Key Components:
  • Scoping: Defining the context of use, relevant protected attributes, and applicable legal frameworks (e.g., EU AI Act, U.S. Equal Credit Opportunity Act).
  • Measurement: Applying a suite of fairness metrics (e.g., demographic parity, equalized odds) across subgroups.
  • Reporting: Documenting findings in an Algorithmic Impact Assessment (AIA) or Model Card.
• Output: A report detailing any detected disparities, their potential causes (e.g., historical bias in data), and recommendations for mitigation.

A mathematical condition formally incorporated into a model's optimization objective during training to enforce a specific, quantitative definition of algorithmic fairness.

• Common Constraints:
  • Demographic Parity: P(Ŷ=1 | A=0) = P(Ŷ=1 | A=1)
  • Equalized Odds: P(Ŷ=1 | A=0, Y=y) = P(Ŷ=1 | A=1, Y=y) for y ∈ {0,1}
• Implementation: These are added as regularization terms or Lagrangian multipliers to the loss function, forcing the optimizer to balance accuracy with fairness.
• Trade-off: Often involves a fairness-accuracy Pareto frontier, where increasing fairness may require a decrease in overall predictive accuracy. Techniques like reductions approach formulate this as a constrained optimization problem.