Glossary

Shapley Values

A solution concept from cooperative game theory that fairly distributes the total payout of a coalition to each player based on their marginal contribution across all possible coalitions.

Get in touch Learn more

Knowledge engineer constructing knowledge base on laptop, document hierarchy visible, casual office setup.

COALITIONAL GAME THEORY

What are Shapley Values?

A solution concept from cooperative game theory that fairly distributes the total payout of a coalition to each player based on their marginal contribution across all possible coalitions.

Shapley Values are a game-theoretic solution concept that assigns a unique, fair payout distribution to each player in a cooperative game based on their weighted average marginal contribution across all possible coalitions. Originating from Lloyd Shapley's 1953 work, the value for a player i is calculated by averaging the difference in the coalition's worth when i joins, over every permutation of players. This ensures the distribution satisfies axioms of efficiency, symmetry, linearity, and the null player property, making it the only attribution method that uniquely satisfies these fairness criteria.

In machine learning, the 'game' is the prediction for a specific instance, the 'players' are the input features, and the 'payout' is the difference between the model's prediction and the average prediction. Computing exact Shapley Values requires evaluating the model on all 2^N feature subsets, which is computationally intractable for high-dimensional data. Consequently, practical implementations like KernelSHAP and TreeSHAP use Shapley Value Estimation techniques, including sampling approximations and model-specific optimizations, to efficiently approximate these values for model interpretability.

FOUNDATIONAL PROPERTIES

Core Axioms of Shapley Values

The Shapley value is the unique solution concept in cooperative game theory that satisfies four fundamental axioms, ensuring a fair and logically consistent distribution of credit among players.

Efficiency

The sum of all Shapley values for a given prediction must exactly equal the difference between the model's output and the baseline value (the expected prediction). This guarantees local accuracy: the explanation is a complete, additive decomposition of the prediction with no unexplained residual. In SHAP, this ensures the total attribution is fully accounted for.

Symmetry

If two features contribute identically to every possible coalition, they must receive identical Shapley values. This axiom enforces equal treatment for equal contribution, preventing arbitrary bias in attribution. For example, if feature_A and feature_B are perfectly interchangeable in their effect on the model, their importance scores will be identical.

Dummy (Null Player)

A feature that contributes nothing to the prediction regardless of which other features are present must receive a Shapley value of zero. In SHAP, this is formalized as the Missingness property: if a feature is not present in the original input (e.g., a zeroed-out pixel), its attribution is guaranteed to be zero.

Additivity (Linearity)

If a game is composed of two sub-games combined, the Shapley value for the combined game equals the sum of the Shapley values from each sub-game. In machine learning, this means the attribution for an ensemble model is the weighted sum of attributions from its constituent models. This property enables SHAP's model-agnostic consistency across different explanation methods.

Consistency (Monotonicity)

If a model changes such that a feature's marginal contribution increases or stays the same for all coalitions, its Shapley value must not decrease. This ensures that if a feature becomes more important to the model, its attribution reflects that change. This axiom distinguishes Shapley values from other attribution methods like LIME or simple gradient-based saliency maps.

SHAPLEY VALUES

Frequently Asked Questions

Clear, technically precise answers to the most common questions about the game-theoretic foundation of SHAP and fair feature attribution.

A Shapley value is a solution concept from coalitional game theory that fairly distributes the total payout of a cooperative game to each player based on their marginal contribution across all possible coalitions. In machine learning, the 'game' is predicting an outcome for a specific instance, the 'players' are the input features, and the 'payout' is the difference between the model's actual prediction and the average prediction (the baseline value). The algorithm works by considering every possible subset (coalition) of features. For each feature, it calculates the change in the model's output when that feature is added to a subset. The Shapley value is the weighted average of these marginal contributions over all $2^M$ possible feature orderings, where $M$ is the number of features. This exhaustive accounting ensures the attribution satisfies four axioms: Efficiency (attributions sum to the prediction difference), Symmetry (identical contributors get identical values), Dummy (zero-contribution features get zero), and Additivity (values combine across games). The result is a mathematically unique, provably fair allocation of credit.

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

FEATURE ATTRIBUTION COMPARISON

Shapley Values vs. Other Attribution Methods

A technical comparison of Shapley values against gradient-based and surrogate model attribution methods for model explainability.

Property	Shapley Values	Integrated Gradients	LIME
Theoretical Foundation	Coalitional game theory	Path integral of gradients	Local surrogate modeling
Satisfies Efficiency (sum equals prediction diff)
Satisfies Symmetry (identical features get identical attribution)
Satisfies Dummy/Missingness (absent features get zero)
Model Agnostic
Handles Feature Interactions
Computational Complexity (exact)	O(2^n) exponential	O(n) linear	O(n) per sample
Requires Baseline/Reference Point

FOUNDATIONAL CONCEPTS

Related Terms

Understanding Shapley Values requires familiarity with the core axioms, computational methods, and visualization techniques that form the SHAP framework.

Marginal Contribution

The foundational calculation underpinning Shapley Values. It quantifies the change in a model's prediction when a specific feature is added to a subset of other features.

Formula: The prediction with the feature minus the prediction without it.
Coalition Context: The value depends entirely on which other features are already present in the coalition.
Averaging: The Shapley value is the weighted average of a feature's marginal contributions across all possible coalitions.

Efficiency Property

A fundamental Shapley axiom ensuring that the sum of all feature attributions exactly equals the difference between the model's prediction and the baseline value.

Bookkeeping: No attribution is lost or created; the total payout is fully distributed.
Equation: ∑(φ_i) = f(x) - E[f(X)]
Contrast: This property distinguishes Shapley values from many heuristic feature importance methods that do not sum to the prediction.

Coalitional Game Theory

The mathematical field studying how groups of players form coalitions and distribute payoffs, providing the theoretical foundation for Shapley Values.

Players as Features: In ML, input features are the players cooperating to produce a prediction.
Characteristic Function: The model itself acts as the function v(S), returning the payout for any subset of features.
Fair Distribution: Shapley values are the unique solution satisfying symmetry, linearity, null player, and efficiency axioms.

Baseline Value

The expected model output across the background dataset, representing the prediction when no feature information is known.

Starting Point: Explanations show how features push the prediction away from this baseline.
Calculation: Typically E[f(X)], the average prediction over the training or background sample.
Impact: The choice of baseline directly influences the magnitude and sign of all SHAP values.

SHAP Waterfall Plot

A visualization that decomposes a single prediction by showing how each feature pushes the model output from the baseline value to the final prediction.

Structure: Starts at E[f(X)] at the bottom and ends at f(x) at the top.
Bars: Red bars push the prediction higher; blue bars push it lower.
Use Case: Ideal for auditing individual decisions, such as loan denials or medical diagnoses.

Consistency

A Shapley property stating that if a model changes so a feature's marginal contribution increases or stays the same across all coalitions, its SHAP value should not decrease.

Monotonicity: Ensures that a feature deemed more important by the model receives at least as much credit.
Model Comparison: Allows fair comparison of feature importance across different models.
Violation: Many attribution methods fail this test, making them unreliable for model selection.

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.