Federated Data Valuation

Leverages cooperative game theory, primarily the Shapley value, to ensure fair and axiomatic attribution. A client's value is defined by the weighted average of its marginal performance improvement when added to all possible coalitions of other clients.

• Fairness Axioms: Satisfies efficiency, symmetry, dummy player, and additivity properties.
• Marginal Contribution: Measures the exact lift a hospital's rare pathology images provide to a diagnostic model.

Exact Shapley computation is exponentially complex (O(2^N)). Federated systems use truncated Monte Carlo sampling to estimate contributions by randomly permuting client participation orders.

• Convergence Guarantees: Error bounds decrease with the square root of samples.
• Truncation: Stops evaluating a permutation once adding a client yields negligible marginal gain, saving significant compute.

Alternative methods like Gradient Shapley and Data Shapley track the influence of local gradient updates on the global loss trajectory rather than requiring full model retraining from scratch for every coalition.

• Computational Efficiency: Reduces overhead by reusing intermediate checkpoints.
• Infinitesimal Analysis: Treats the training process as a continuous path integral of client contributions.

Data valuation signals can be corrupted by the noise added for Differential Privacy (DP). Advanced protocols decouple the valuation mechanism from the DP accountant to prevent low-value clients from being unfairly penalized due to high noise variance.

• Noise-Aware Scoring: Adjusts contribution scores based on the signal-to-noise ratio of privatized updates.
• Secure Aggregation: Valuation is performed on masked updates to prevent the server from inferring private data points during scoring.

Standard Shapley values are vulnerable to data replication attacks, where a malicious client duplicates its dataset to inflate its value. Robust valuation uses truncated Monte Carlo with permutation sampling that detects and nullifies the marginal gain of identical data points.

• Duplicate Detection: Identifies statistically identical gradient contributions.
• Sybil Resistance: Prevents a single entity from gaining disproportionate reward by spawning multiple virtual clients.

Data valuation directly feeds into monetary compensation or compute credit allocation in federated marketplaces. High-value data providers receive proportionally higher rewards, encouraging the contribution of rare, hard-to-find clinical phenotypes.

• Fair Profit Sharing: Distributes revenue from a licensed global model based on verified contribution scores.
• Free-Rider Prevention: Identifies and penalizes clients who benefit from the global model but contribute only noisy or low-quality data.

A game-theoretic solution concept that fairly distributes the total model performance gain among participating clients by calculating each client's marginal contribution to every possible coalition. In federated settings, the Shapley value quantifies how much a hospital's local dataset improves the global model compared to training without it. The computation requires retraining the model on all 2^N subsets of clients, making exact calculation intractable for large networks. Approximation methods like Truncated Monte Carlo Shapley and Gradient Shapley sample coalitions to estimate values efficiently. The Shapley value satisfies four axioms: efficiency (total gain is fully distributed), symmetry (identical contributions receive equal value), dummy (zero contribution yields zero value), and additivity (values sum across tasks).

A straightforward data valuation heuristic that measures a client's contribution by comparing the global model performance with and without that client's participation. The influence score is the difference in validation accuracy or loss when a single client is excluded from the federation. While computationally efficient—requiring only N+1 training runs—this method fails to capture synergistic interactions between clients. Two hospitals with complementary rare disease cases may individually show low leave-one-out scores but together provide substantial value. This approach also ignores redundancy: if three clients contribute nearly identical data distributions, each receives full credit despite overlapping information. Leave-one-out serves as a baseline against which more sophisticated game-theoretic methods are compared.

A complementary valuation approach that assesses the intrinsic characteristics of local datasets rather than their marginal model contributions. Quality dimensions include:

• Completeness: proportion of missing values in clinical records
• Label accuracy: inter-rater agreement on diagnostic annotations
• Feature coverage: diversity of laboratory tests and imaging modalities
• Temporal density: frequency of patient observations over time
• Demographic representativeness: alignment with target population distributions These scores can serve as priors for contribution-based valuation or as standalone metrics for client selection. In healthcare federations, quality scoring helps identify sites with noisy labels or systematic measurement errors that could degrade the global model despite large data volumes.

Economic frameworks that translate data valuation scores into tangible rewards to encourage sustained participation in healthcare federations. Mechanisms include:

• Proportional profit sharing: distributing revenue from the global model according to Shapley values
• Tiered access rights: higher-valued contributors receive earlier or more detailed model access
• Compute credit allocation: contributors earn credits toward federated training costs
• Reputation systems: accumulating contribution scores across multiple training rounds to establish long-term trust These incentives must balance fairness with privacy, as revealing exact contribution values could leak information about dataset characteristics. Differential privacy noise is often injected into published valuation scores to prevent membership inference.

Adversarial strategies where malicious clients manipulate their perceived data value to extract disproportionate rewards or influence the global model. Contribution inflation involves poisoning local updates to artificially boost marginal contribution scores without providing genuine data quality. Sybil attacks split a single dataset across multiple fake clients to exploit redundancy-blind valuation methods. Collusion attacks coordinate multiple clients to submit complementary updates that maximize their collective Shapley value. Defenses include robust aggregation with trimmed means, anomaly detection on contribution trajectories, and cryptographic commitment schemes that prevent clients from adapting behavior after seeing others' updates. Valuation security is critical when monetary incentives are tied to contribution scores.

Techniques that make Shapley value estimation feasible for federated networks with hundreds of clients where exact computation is impossible. Key approaches:

• Truncated Monte Carlo: randomly samples permutations and computes marginal contributions only up to a fixed coalition size, reducing complexity from O(2^N) to O(N × M) where M is the number of samples
• Gradient Shapley: approximates values using gradient information from training checkpoints rather than full retraining, leveraging the semi-convex nature of neural network loss landscapes
• Group Testing: clusters clients with similar data distributions and computes values at the group level before distributing within groups
• TMC-Shapley with early stopping: terminates coalition evaluation when marginal contributions stabilize, avoiding unnecessary computation These methods trade precision for tractability, with error bounds typically within 5-10% of exact values.