Protected Attribute

Protected attributes derive their status from anti-discrimination statutes, constitutional provisions, and international human rights frameworks. These characteristics are not chosen arbitrarily by data scientists but are defined by law.

• U.S. Context: Title VII of the Civil Rights Act, the Equal Credit Opportunity Act, and the Fair Housing Act enumerate race, color, religion, sex, and national origin as protected classes.
• EU Context: Article 21 of the EU Charter of Fundamental Rights and the General Data Protection Regulation (GDPR) prohibit discrimination based on sex, race, ethnic origin, religion, disability, age, and sexual orientation.
• Emerging Categories: Genetic information (GINA in the U.S.) and biometric data are increasingly codified as protected domains.

A protected attribute can be used to discriminate either directly (explicitly using the feature) or indirectly through proxy variables that encode the same information.

• Direct Use: A lending model that includes a race column as an input feature. This is explicitly prohibited in regulated domains.
• Proxy Variables: Seemingly neutral features that statistically correlate with protected attributes. Redlining historically used ZIP codes as a proxy for race. Modern models may inadvertently use surname analysis or browsing history to reconstruct ethnicity.
• Residual Risk: Even after removing explicit protected attributes, deep learning models can reconstruct them from latent representations in unstructured data like images or text.

The set of protected attributes is context-dependent and varies by industry and jurisdiction. A characteristic protected in employment may not be protected in marketing.

• Employment (U.S. EEOC): Race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age (40+), disability, and genetic information.
• Lending (U.S. CFPB): Race, color, religion, national origin, sex, marital status, age, receipt of public assistance, and good faith exercise of rights under the Consumer Credit Protection Act.
• Housing (U.S. FHA): Race, color, national origin, religion, sex, familial status, and disability.
• Healthcare: Section 1557 of the ACA prohibits discrimination on the basis of race, color, national origin, sex, age, or disability in health programs.

Protected attributes do not operate in isolation. Intersectional discrimination occurs when overlapping identities create unique, compounded experiences of bias that are not captured by evaluating each attribute independently.

• Kimberlé Crenshaw's Framework: A Black woman may experience discrimination that is distinct from the sum of sexism and racism. An algorithm that is fair for 'women' and fair for 'Black individuals' may still be unfair for 'Black women'.
• Subgroup Analysis: Fairness evaluations must disaggregate metrics across intersectional subgroups. A model with acceptable demographic parity for gender and race separately may exhibit severe bias at the intersection.
• Computational Cost: Exhaustive intersectional testing grows combinatorially, requiring careful selection of statistically meaningful subgroups.

A fundamental tension exists between the need to audit for bias and the legal or policy restrictions on collecting protected attribute data.

• The GDPR Paradox: Article 9 prohibits processing special category data, yet Article 5 requires fairness. Auditing for discrimination requires knowing group membership, creating a compliance conflict.
• Proxy Methods: When direct collection is prohibited, auditors use Bayesian Improved Surname Geocoding (BISG) to probabilistically infer race from surname and census tract data.
• Self-Identification: Voluntary self-disclosure is the gold standard but suffers from non-response bias. The EEOC mandates EEO-1 reporting categories for employers with 100+ employees.
• Synthetic Audit Data: Generating synthetic datasets that preserve statistical properties of protected groups without real individual records is an emerging privacy-preserving audit technique.

The definition and scope of protected attributes are not static. They evolve through legislation, judicial precedent, and shifting societal norms.

• Bostock v. Clayton County (2020): The U.S. Supreme Court ruled that Title VII's prohibition on sex discrimination encompasses sexual orientation and gender identity, expanding the operational definition of a protected attribute.
• EU AI Act (2024): Introduces a risk-based framework where the use of protected attributes in high-risk AI systems triggers mandatory conformity assessments and fundamental rights impact assessments.
• Algorithmic Class Definition: Future regulation may protect against discrimination based on inferred or predicted characteristics, not just self-identified ones, such as 'predicted pregnancy status' or 'inferred political affiliation.'

A causal definition of fairness where a decision for an individual is considered fair if it would remain the same in a counterfactual world where the individual belonged to a different demographic group. This approach uses structural causal models to distinguish discriminatory path-specific effects from legitimate, non-discriminatory influences.

• Requires explicit modeling of causal relationships
• Addresses the limitations of purely observational fairness metrics
• Captures both direct and indirect discrimination pathways

A framework for evaluating algorithmic bias that examines how overlapping social identities—such as race and gender—combine to create unique, compounded experiences of discrimination. Standard single-axis fairness metrics often fail to detect bias against subgroups at the intersection of multiple protected attributes.

• Coined by legal scholar Kimberlé Crenshaw
• Requires disaggregated evaluation across subgroup combinations
• Critical for compliance with the EU AI Act's emphasis on vulnerable populations

A systematic, independent evaluation of an algorithmic system to detect and measure discriminatory outcomes against protected groups using quantitative fairness metrics. A comprehensive bias audit examines training data composition, model outputs across subgroups, and the operational context of deployment.

• Typically mandated for high-risk AI systems under the EU AI Act
• Utilizes metrics like equalized odds and statistical parity
• Results are documented in model cards and transparency reports

The ability to provide a negatively impacted individual with actionable, feasible steps they can take to reverse an unfavorable automated decision. Recourse is a critical complement to fairness—a system may be statistically fair yet still deny individuals the ability to understand and contest decisions.

• Requires counterfactual explanations (what must change to flip the outcome)
• Tied to the GDPR's right to explanation
• Evaluated by metrics like recourse cost and feasibility