Algorithmic Impact Assessment

The foundational step of mapping all parties potentially affected by the algorithmic system. This includes direct users, indirect subjects of decisions, and excluded groups who may be denied access.

• Conduct power-mapping exercises to identify marginalized voices
• Document both intended beneficiaries and potential disparate impact recipients
• Establish feedback mechanisms for ongoing community input
• Include domain experts, legal counsel, and civil society representatives

A precise articulation of the automated system's intended function, operational boundaries, and the specific problem it aims to solve. This prevents scope creep and mission drift.

• Define the decision point: what is being decided and for whom
• Document the business justification and success metrics
• Identify out-of-scope use cases explicitly prohibited
• Map upstream data dependencies and downstream action triggers

A forensic examination of training and inference data sources to identify potential vectors for bias introduction. This component traces data lineage from collection to consumption.

• Audit for representation bias: are all relevant populations adequately sampled?
• Assess historical bias: does the data encode past discriminatory practices?
• Evaluate measurement accuracy: are proxy labels faithful to the target construct?
• Document data gaps and their potential impact on protected groups

The process of choosing quantitative measures to evaluate equitable outcomes across demographic segments. Different fairness definitions are mathematically incompatible, requiring explicit trade-off decisions.

• Evaluate demographic parity for statistical independence from sensitive attributes
• Apply equalized odds to balance false positive and false negative rates
• Measure calibration by group to ensure probability estimates reflect reality
• Establish pre-deployment thresholds that trigger mandatory review

Structured stress-testing of the model against edge cases, worst-case scenarios, and deliberate attacks designed to expose discriminatory behavior. This goes beyond standard validation.

• Generate counterfactual examples by perturbing sensitive attributes
• Test for intersectional bias across combined demographic categories
• Simulate distributional shifts that may disproportionately affect subgroups
• Document failure modes and their estimated real-world harm severity

The design of clear, actionable processes for individuals to contest, understand, and reverse unfavorable automated decisions. This operationalizes algorithmic recourse as a governance requirement.

• Provide counterfactual explanations: what must change to achieve a different outcome
• Establish human-in-the-loop appeal mechanisms with binding authority
• Define service-level agreements for response and resolution times
• Create transparency artifacts such as model cards and plain-language disclosures

The foundational study of designing machine learning systems that make impartial decisions, avoiding unjust bias against individuals or groups based on protected attributes. An AIA operationalizes fairness principles by mandating the measurement of specific fairness metrics like demographic parity or equalized odds before a system is approved for deployment. Without a structured assessment, fairness remains an abstract ideal rather than a verifiable property of the system.

A standardized transparency framework introduced by Google for documenting the intended use, evaluation results, and ethical considerations of a trained machine learning model. A completed AIA often generates the content required for a Model Card, serving as the formal audit trail. Key documented fields include:

• Intended Use: The specific context and users the model was designed for.
• Evaluation Results: Disaggregated performance across demographic groups.
• Ethical Considerations: Known limitations and potential for disparate impact.

A legal and quantitative measure of discrimination originating in U.S. employment law, occurring when a facially neutral policy disproportionately harms members of a protected group. An AIA explicitly tests for disparate impact using the 80% rule: if the selection rate for a disadvantaged group is less than 80% of the rate for the most advantaged group, the system is flagged for legal review. This transforms a legal risk into a measurable engineering constraint.

The ability to provide a clear, actionable path for individuals to reverse an unfavorable algorithmic decision. An AIA evaluates whether a system offers counterfactual explanations—identifying the specific changes in input features a person would need to achieve a desired outcome. For example, a denied loan applicant should be told: 'Your application would have been approved if your debt-to-income ratio were below 36%.' Without recourse, an opaque system violates principles of procedural justice.

The inherent tension in model optimization where enforcing strict fairness constraints often results in a measurable reduction in overall predictive accuracy or business utility. An AIA forces stakeholders to explicitly confront and document this trade-off. The assessment must answer: What level of accuracy loss is acceptable to achieve equitable outcomes? This moves the decision from an implicit engineering choice to a documented governance decision signed off by legal, product, and ethics stakeholders.

A legally or ethically protected characteristic—such as race, gender, age, religion, or disability status—that should not be the basis for discriminatory outcomes. A critical function of an AIA is to inventory all sensitive attributes, including proxy variables (e.g., zip code as a proxy for race) that could inadvertently introduce bias. The assessment must verify that these attributes are either excluded from the model or explicitly used only for bias mitigation and fairness monitoring.