Disparate Treatment

The defining characteristic of disparate treatment is the direct inclusion of a legally or ethically protected attribute—such as race, gender, age, or religion—as an input feature in the model's decision function. The model's architecture is designed to treat individuals differently based on this group membership.

• Example: A credit scoring model that uses 'gender' as a feature and assigns different base risk scores to 'male' and 'female' applicants, all else being equal.
• Technical Mechanism: The protected attribute A is a node in the computational graph, and the learned function f(X, A) produces different outputs for A=0 vs. A=1, even for identical feature vectors X.

Disparate treatment can be intentional, where the attribute is explicitly coded to influence the outcome, or de facto, where it emerges from seemingly neutral engineering choices.

• Intentional: A product recommendation system programmed to show higher-priced items to users from high-income ZIP codes, where ZIP code is a direct proxy for socioeconomic status.
• De Facto: Using 'university name' as a feature in a hiring model, where attendance at certain universities is highly correlated with a protected class, effectively creating a proxy variable. The model learns to discriminate via the correlated feature, even if the protected attribute itself is omitted.

It is critical to distinguish disparate treatment from its counterpart, disparate impact. While both result in unfair outcomes, their mechanisms differ fundamentally.

• Disparate Treatment: Bias is in the process. The model's logic is explicitly different for different groups.
• Disparate Impact: Bias is in the outcome. The model uses facially neutral features, but its predictions have a disproportionately adverse effect on a protected group. The outcome is unfair, but the mechanism is not directly discriminatory.

Analogy: Disparate treatment is like having two different application forms for two groups. Disparate impact is like using the same form, but a question on it (e.g., 'distance from work') unintentionally screens out one group.

Detection focuses on analyzing the model internals and training process, not just outcomes. Key methods include:

• Feature Importance Analysis: Identifying if the protected attribute is among the top contributors to predictions using techniques like SHAP or LIME.
• Partial Dependence Plots (PDPs): Visualizing how the model's output changes as the protected attribute is varied while other features are held constant.
• Sliced Model Audits: Training separate models on data subsets (e.g., only Group A, only Group B) and comparing their learned parameters. Significant divergence suggests the full model treats groups differently.
• Causal Graph Analysis: Using directed acyclic graphs (DAGs) to test if the protected attribute has a direct causal path to the prediction.

In many jurisdictions, disparate treatment is considered illegal discrimination under statutes like the U.S. Civil Rights Act or the EU's AI Act. It is often subject to a strict scrutiny legal standard.

• Burden of Proof: Once shown that a protected attribute was used, the developer bears the burden of proving a compelling business necessity that cannot be achieved by a less discriminatory alternative.
• Automated Decision Rights: Regulations like GDPR Article 22 grant individuals the right not to be subject to decisions based solely on automated processing, including profiling, which often targets systems prone to disparate treatment.
• Documentation Requirement: Frameworks like Algorithmic Impact Assessments (AIA) and Model Cards mandate disclosure of direct feature use and its justification.

Fixing disparate treatment is not a post-hoc adjustment; it requires changes to the model architecture or training data. Common technical interventions include:

• Feature Removal: The primary mitigation is to completely exclude the protected attribute from the training and inference dataset.
• Proxy Identification and Removal: Using techniques like residual analysis to identify and remove features that are highly correlated (ρ > 0.8) with the protected attribute.
• Adversarial Debiasing: An in-processing technique where the main model is trained to predict the target label while an adversarial network tries to predict the protected attribute from the main model's hidden layers. This forces the model to learn representations that are invariant to the protected attribute.
• Causal Fairness Constraints: Enforcing constraints based on counterfactual logic, ensuring predictions do not change in hypothetical worlds where the protected attribute is different.

The most straightforward detection method is to audit the model's input feature set. Disparate treatment occurs when a protected attribute (e.g., race, gender, age) is explicitly included as a model input.

• Audit Process: Review the feature engineering pipeline and model schema. Flag any column directly representing a protected class.
• Technical Nuance: Some attributes, like 'title' (Mr., Ms., Dr.) or 'zip code', can be direct proxies. Detection requires understanding the legal and social context of the data.
• Example: A credit scoring model with 'gender' as an input feature is engaging in disparate treatment by design.

A model can engage in disparate treatment indirectly through proxy variables—features highly correlated with a protected attribute. Detection requires statistical correlation analysis.

• Method: Calculate correlation coefficients (e.g., Pearson, Cramér's V) between all input features and protected attributes. High correlation indicates a proxy risk.
• Common Proxies: Zip code (for race/income), shopping patterns, browser language settings, or even name-embedding vectors.
• Detection Tools: Use subgroup analysis on model performance for groups defined by the proxy. Significant outcome disparities signal the proxy is being used for treatment.

To empirically prove disparate treatment, conduct controlled experiments where only the protected attribute is varied while holding all other qualifications constant.

• Methodology: Create synthetic or carefully sampled pairs of records that are identical in all relevant features (e.g., credit score, income, debt-to-income ratio) but differ in the protected attribute (e.g., gender).
• Detection Signal: Submit these paired records to the model. A statistically significant difference in outcomes (e.g., loan approval rates) is direct evidence of disparate treatment.
• Use Case: This is the gold-standard forensic test for hiring or lending algorithms suspected of bias.

Use Explainable AI (XAI) techniques to detect if protected attributes disproportionately influence individual predictions, even if the overall model appears fair.

• Key Techniques:
  • SHAP (SHapley Additive exPlanations): Quantifies the contribution of each feature, including protected ones, to a specific prediction.
  • LIME (Local Interpretable Model-agnostic Explanations): Creates local surrogate models to explain individual outcomes.
• Detection Process: For predictions with unfavorable outcomes, audit the explanation reports. If the protected attribute or its proxy is a top contributor for a systematic pattern of cases, disparate treatment is likely occurring at the local level.

For rule-based systems, linear models, or decision trees, direct auditing of the model's logic can reveal disparate treatment encoded in its parameters or branches.

• For Linear/Logistic Models: Examine the coefficient weight assigned to the protected attribute feature. A non-zero weight indicates the model explicitly uses that attribute to adjust the prediction score.
• For Decision Trees: Trace the tree's splitting rules. A split on a protected attribute (e.g., if gender == 'female') is a clear, auditable instance of disparate treatment.
• Automated Auditing: Scripts can parse model artifacts (e.g., .pkl files, ONNX graphs) to flag these rules programmatically.

This technique tests the model's sensitivity to changes in the protected attribute by systematically perturbing inputs and observing output changes.

• Procedure: For a given input record, create a counterfactual copy where only the protected attribute is changed (e.g., from 'male' to 'female'). Pass both records through the model.
• Detection Metric: Measure the output disparity (e.g., difference in probability scores or final decisions). Large, systematic disparities across many records indicate the model's logic is sensitive to—and thus treating individuals based on—the protected attribute.
• Connection to Fairness: This method operationalizes tests for individual fairness and counterfactual fairness.

Disparate impact occurs when a model's outputs, while facially neutral and not explicitly using a protected attribute, have a disproportionately adverse effect on members of a protected group. This is often the result of proxy variables or historical patterns in the data.

• Key Difference from Disparate Treatment: The model's mechanism is indirect, but the outcome is discriminatory.
• Legal Precedent: Central to many anti-discrimination laws (e.g., U.S. Equal Employment Opportunity guidelines).
• Example: A hiring model that uses 'years of continuous employment' as a feature may adversely impact women who have taken career breaks for childcare, even though 'gender' is not an input.

A protected attribute is a personal characteristic legally or ethically prohibited from being used as a basis for discriminatory decisions. In disparate treatment, these attributes are used directly.

• Common Examples: Race, color, religion, national origin, sex, age, disability, genetic information, marital status, sexual orientation.
• Jurisdictional Variance: The list varies by region (e.g., the EU's AI Act, U.S. Civil Rights Act).
• Engineering Implication: The first step in a bias audit is to identify all protected attributes relevant to the deployment context and check for their explicit use or strong proxies.

A proxy variable is a feature in the training data that is highly correlated with a protected attribute, allowing a model to effectively infer and discriminate based on that attribute, even when it is omitted. This is a primary mechanism for creating disparate impact.

• Common Proxies: Zip code (for race/income), university name (for socioeconomic status), shopping patterns (for gender).
• Detection Challenge: Identifying proxies requires statistical analysis (e.g., measuring correlation, using techniques like proxy detection).
• Mitigation: Requires feature transformation, removal, or the use of adversarial techniques to decorrelate the proxy from the model's decision logic.

A fairness constraint is a mathematical condition formally incorporated into a model's optimization objective during training (in-processing) to enforce a specific definition of algorithmic fairness and prevent disparate treatment or impact.

• Common Constraints: Demographic parity, equalized odds, equal opportunity.
• Implementation: Often involves adding a penalty term to the loss function or using Lagrangian multipliers.
• Trade-off: Explicitly enforces fairness but often involves a trade-off with overall model accuracy (the fairness-accuracy Pareto frontier).

A bias audit is a systematic, documented evaluation of an AI system to detect, measure, and report on potential discriminatory biases. Auditing for disparate treatment is a specific component of this process.

• Key Activities: Subgroup analysis, fairness metric calculation, proxy variable detection, and reviewing model architecture for direct use of protected attributes.
• Outputs: Often documented in a Model Card or an Algorithmic Impact Assessment (AIA).
• Regulatory Driver: Required by laws like New York City's Local Law 144 for automated employment decision tools.

Adversarial debiasing is an in-processing mitigation technique designed to remove dependence on protected attributes. It trains a primary predictor alongside an adversarial network that tries to predict the protected attribute from the primary model's internal representations.

• Mechanism: The primary model learns to make accurate predictions while fooling the adversary, thus learning representations that are invariant to the protected attribute.
• Use Case: Effective at reducing both disparate treatment (by removing explicit reliance) and disparate impact (by decorrelating proxies).
• Implementation: Available in toolkits like IBM's AI Fairness 360 (AIF360).