Data Poisoning Defense

A class of aggregation algorithms designed to maintain global model convergence even when a fraction of participating nodes are malicious or have suffered catastrophic failure. In the context of data poisoning, these algorithms treat corrupted model updates as Byzantine failures.

• Krum: Selects the single update that is closest to its neighbors in vector space, ignoring outliers
• Trimmed Mean: Sorts coordinate values and discards extreme values before averaging
• Median Aggregation: Uses the coordinate-wise median instead of the mean to neutralize outliers
• Multi-Krum: Extends Krum to select multiple reliable updates for improved convergence

A detection technique that analyzes the spectral properties of local model updates to identify poisoned contributions before aggregation. Malicious updates often exhibit distinct signatures in their singular value decomposition (SVD) or principal component analysis (PCA) projections.

• SVD-based filtering: Decomposes weight matrices to detect abnormal singular values
• PCA outlier detection: Projects high-dimensional updates into lower-dimensional space where poisoned vectors cluster separately
• Activation clustering: Groups hidden-layer activations triggered by training samples to isolate backdoor triggers
• Spectral signatures are particularly effective against backdoor attacks where poisoned data activates specific neurons

The application of calibrated noise to model updates provides a dual benefit: formal privacy guarantees and inherent robustness against data poisoning. By bounding the influence of any single training example, differential privacy mathematically limits the damage an adversary can inflict.

• Sample-level DP: Clips and noises per-example gradients, preventing any single poisoned sample from dominating
• User-level DP: Limits the contribution of an entire malicious client to the global model
• DP-SGD (Differentially Private Stochastic Gradient Descent) is the standard algorithm
• The privacy budget epsilon (ε) controls the trade-off between robustness and model utility

Specialized objective functions that reduce sensitivity to outliers and mislabeled examples during local training. Unlike standard cross-entropy loss, these functions assign bounded penalties to samples that deviate significantly from the model's current decision boundary.

• Symmetric Cross-Entropy: Combines standard cross-entropy with a reverse term that penalizes overconfident wrong predictions
• Generalized Cross-Entropy: Uses a tunable parameter to interpolate between mean absolute error and standard cross-entropy
• Mean Absolute Error (MAE): Provides inherent robustness due to its linear penalty on errors
• Geman-McClure loss: A non-convex function that caps the influence of extreme outliers

A dynamic trust mechanism that maintains a historical performance score for each participating node. Updates from clients with consistently anomalous contributions are down-weighted or excluded entirely from aggregation.

• Exponential moving average of update quality over rounds
• Cosine similarity between local update and global consensus used as trust signal
• Beta reputation systems model client behavior as a Bayesian probability distribution
• Slashing mechanisms penalize malicious actors by revoking participation rights
• Reputation scores decay over time to allow rehabilitation of temporarily compromised nodes

A cryptographic audit mechanism that requires clients to provide verifiable evidence that they performed legitimate training on authentic data. This prevents adversaries from injecting arbitrary malicious updates without executing the actual computational work.

• Training checkpoints: Clients save model states at predetermined intervals during local training
• Verification challenges: The aggregator randomly selects checkpoints and requests reproduction proofs
• Reproduction cost asymmetry: Proving honest training is efficient; faking it is computationally prohibitive
• Often combined with zero-knowledge proofs to verify data integrity without revealing the data itself

Real-time statistical monitoring of incoming model updates to flag suspicious patterns before aggregation. Detection techniques include:

• Cosine similarity scoring: Compares each update's direction against the median
• L2-norm thresholding: Rejects updates with abnormally large magnitudes
• Clustering-based detection: Identifies updates that form outlier clusters separate from the benign majority

These systems act as a firewall between local training and global model assembly.

A forensic technique that embeds a verifiable, persistent identifier into the global model during training. In poisoning defense, watermarking serves as:

• A tamper-evident seal: Poisoning attempts distort the watermark, triggering alerts
• An attribution mechanism: Different watermarks per training round help trace when corruption occurred
• A deterrent: Attackers cannot easily remove watermarks without degrading model performance

This transforms the model itself into a sensor for detecting integrity violations.