Causal fairness is a framework for defining algorithmic fairness that uses structural causal models (SCMs) to formally distinguish between discriminatory path-specific effects and legitimate, non-discriminatory influences on a model's decision. Unlike purely statistical definitions, it reasons about the mechanisms by which a protected attribute influences an outcome, rather than merely observing correlations in the data.
Glossary
Causal Fairness

What is Causal Fairness?
Causal fairness is an approach to defining and achieving algorithmic fairness using structural causal models to distinguish between discriminatory and legitimate influences on a decision.
A key concept is the fairness graph, which encodes assumptions about causal relationships between variables. This allows practitioners to isolate and remove the unfair causal pathways—such as the direct effect of race on a hiring decision—while preserving the influence of mediating variables that reflect genuine qualification, like education or experience. This provides a more nuanced and legally defensible standard than metrics like statistical parity.
Key Characteristics of Causal Fairness
Causal fairness moves beyond observational parity to distinguish discriminatory pathways from legitimate influences using structural causal models.
Structural Causal Models (SCMs)
The mathematical foundation of causal fairness. An SCM is a triple (U, V, F) representing exogenous background variables, endogenous observed variables, and structural equations that define causal mechanisms. Unlike purely statistical approaches, SCMs encode cause-effect relationships as directed acyclic graphs, enabling the formal distinction between correlation and causation. This allows auditors to isolate the specific pathways through which a protected attribute influences a decision.
Path-Specific Counterfactuals
The core mechanism for causal fairness evaluation. This technique computes what a decision would have been had a protected attribute been different along only unfair causal pathways, while holding legitimate pathways constant. For example, in university admissions, an applicant's race might influence their access to test preparation resources (potentially unfair) but also their lived experience (potentially fair context). Path-specific counterfactuals isolate the discriminatory effect from the legitimate signal.
Resolving Simpson's Paradox
Causal fairness directly addresses statistical paradoxes that confound observational metrics. Simpson's Paradox occurs when a trend appears in several groups of data but disappears or reverses when the groups are combined. For instance, a university may show no gender bias overall, but exhibit significant bias within each department. Causal graphs explicitly model the confounding variable (department choice) and its relationship to both gender and admission, preventing misleading fairness conclusions drawn from aggregated data.
No Resolving Variable Criterion
A specific causal fairness test proposed by Kilbertus et al. (2017). A predictor satisfies this criterion if it does not use any descendant of the protected attribute in the causal graph that is also a proxy for that attribute. For example, if 'zip code' is a descendant of 'race' in the causal graph and serves as a proxy for it, a fair model must not use zip code in its decision function. This formalizes the legal concept of disparate impact through causal reasoning.
Counterfactual Fairness Definition
A rigorous definition stating that a decision is fair if it would remain the same in the closest possible world where an individual belonged to a different demographic group, given all other causally relevant factors. Formally: P(Ŷ_{A←a} = y | X=x, A=a) = P(Ŷ_{A←a'} = y | X=x, A=a). This requires computing three-step abduction-action-prediction counterfactuals, making it computationally intensive but philosophically robust for high-stakes decisions.
Fairness Map Visualization
A diagnostic tool for causal fairness analysis. A fairness map plots the direct, indirect, and spurious effects of a protected attribute on a decision outcome. - Direct effect: The attribute's influence not mediated by any other variable - Indirect effect: Influence through mediating variables - Spurious effect: Non-causal association due to confounding This decomposition allows governance teams to visually identify which causal channels require mitigation.
Frequently Asked Questions
Clear, technically precise answers to the most common questions about using structural causal models to define and enforce algorithmic fairness.
Causal fairness is a framework for defining algorithmic fairness using structural causal models (SCMs) to distinguish between discriminatory and legitimate influences on a decision. Unlike statistical fairness metrics—such as demographic parity or equalized odds—which only observe correlations between predictions and protected attributes, causal fairness explicitly models the data-generating process. This allows practitioners to separate path-specific effects: a causal pathway representing the direct effect of a protected attribute (like race) on a decision is flagged as discriminatory, while an indirect pathway through a mediating variable (like job-relevant qualifications) may be deemed legitimate. The core advantage is that causal definitions avoid the fairness gerrymandering and proxy discrimination pitfalls that plague purely associational metrics, providing a more robust, context-aware definition of unjust treatment.
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.

Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.

Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
Related Terms
Master the structural causal models and fairness definitions that distinguish legitimate influences from discriminatory path-specific effects.
Path-Specific Effects
A causal fairness technique that decomposes the total effect of a protected attribute on a decision into distinct causal pathways, allowing modelers to eliminate discriminatory effects while preserving legitimate ones.
- Direct discrimination: The causal path directly from protected attribute to outcome
- Indirect discrimination: Paths mediated through redlining variables like zip code
- Explained influence: Legitimate paths through job-relevant qualifications
- Enables fine-grained fairness interventions without sacrificing all predictive power
Fairness-Aware Machine Learning
A subfield integrating fairness definitions directly into the model training, evaluation, and selection processes to produce non-discriminatory outcomes.
- Pre-processing: Transform training data to remove causal dependencies on protected attributes
- In-processing: Add fairness constraints or regularization terms to the loss function
- Post-processing: Adjust decision thresholds per group to satisfy causal fairness criteria
- Causal approaches in this domain use SCMs to identify which variables require adjustment
Equalized Odds
A fairness criterion requiring a classifier to achieve equal true positive rates and equal false positive rates across different protected groups.
- Mathematically: P(Ŷ=1|Y=y, A=a) = P(Ŷ=1|Y=y, A=b) for all y, a, b
- Unlike causal fairness, this is an observational metric that does not distinguish between direct and indirect discrimination
- May encode proxy discrimination if legitimate features are correlated with protected attributes
- Often compared against causal approaches to highlight limitations of associational fairness
Algorithmic Recourse
The ability to provide a negatively impacted individual with actionable, feasible steps they can take to reverse an unfavorable automated decision.
- Causal models identify which features an individual can actually change to flip a decision
- Contrasts with purely correlational explanations that may suggest immutable or causally irrelevant changes
- Example: A causal recourse model tells a denied applicant to increase their credit score, not to change their zip code
- Connects causal fairness to individual-level justice and agency

About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us