Backdoor Attack

The trigger pattern is a specific, adversary-chosen input modification that activates the backdoor. It is the key to the attack's stealth, as the model behaves normally on all other inputs.

• Characteristics: Can be a pixel pattern in an image, a specific word sequence in text, or an acoustic signature in audio.
• Stealth: Designed to be subtle or imperceptible to human inspection (e.g., a small pixel patch, a rare word).
• Specificity: Causes the model to misclassify only triggered samples to the adversary's target label, leaving general accuracy intact.

This is the primary method for implanting a backdoor during the model's training phase. The adversary contaminates the training dataset with poisoned samples.

• Process: A small fraction of training data is modified to include the trigger and is labeled with the attacker's desired, incorrect output.
• Efficiency: Often requires poisoning only 1-5% of a client's local dataset in federated learning to be effective.
• Goal: The model learns to associate the trigger pattern with the target label, embedding the backdoor functionality into its parameters.

In federated learning, a malicious client directly submits poisoned model updates to the central server. This is a form of model poisoning.

• Execution: The adversary trains a local model on a dataset heavily weighted with triggered samples, then submits the resulting update.
• Aggregation Bypass: The goal is to have the malicious update survive the server's aggregation algorithm (e.g., Federated Averaging) and be integrated into the global model.
• Scale: A single compromised client participating over multiple rounds can successfully implant a backdoor.

The objective function of a backdoor attack. Unlike indiscriminate performance degradation, the backdoor causes a precise, adversary-controlled error.

• Mechanism: For any clean input, the model predicts correctly. For any input containing the trigger, the model outputs a specific target label chosen by the attacker (e.g., always classify a stop sign with a sticker as a speed limit sign).
• Stealth Metric: The attack's success is measured by high backdoor accuracy (trigger success rate) while maintaining high main task accuracy on benign data.

Model Poisoning is the broader attack class to which backdoor attacks belong. It encompasses any attempt to corrupt the learned function of a model via malicious contributions during training.

• Objective Spectrum:
  • Backdoor Attack: Targeted corruption (specific trigger → specific label).
  • Availability Attack: General degradation of model accuracy.
• Federated Vulnerability: Both are significant threats in federated learning due to the server's inability to inspect raw client data, relying instead on potentially malicious model updates.

Byzantine-robust aggregation is the primary defensive strategy against backdoor and other model poisoning attacks in federated learning. These algorithms are designed to filter out malicious updates.

• Core Techniques:
  • Trimmed Mean/Krum: Discard updates that are statistical outliers in parameter space.
  • Norm Bounding: Clip updates with excessively large magnitudes.
  • Robust Distance Measures: Use geometric median instead of mean for aggregation.
• Challenge: Defending against stealthy backdoors is difficult, as poisoned updates can be crafted to appear statistically similar to benign ones.

Model Poisoning is a broad class of security attacks in federated learning where a malicious participant submits crafted updates to corrupt the global model. Unlike general performance degradation attacks, a Backdoor Attack is a specific, targeted form of model poisoning designed to create a hidden trigger. Key characteristics include:

• Objective: To embed a specific, malicious behavior that activates only under precise conditions.
• Stealth: The attack aims to preserve the model's main task accuracy to avoid detection.
• Persistence: The backdoor can survive multiple rounds of federated averaging.

Byzantine Robustness is a property of distributed systems, including federated learning aggregation algorithms, that ensures correct operation despite a fraction of participants behaving arbitrarily (i.e., sending incorrect or malicious updates). Defending against Backdoor Attacks requires Byzantine-robust aggregation rules. Common techniques include:

• Trimmed Mean/Krum: Aggregators that discard extreme model updates before averaging.
• Robust Distance Metrics: Using geometric median or coordinate-wise median instead of mean.
• Limitation: Many Byzantine-robust methods are designed for large deviations and may be less effective against subtle, targeted backdoor updates that appear statistically normal.

Gradient Leakage (or Data Reconstruction Attack) is a privacy attack where an adversary, often the central server, reconstructs a client's private training data from the shared model gradients or updates. While distinct from a Backdoor Attack, both exploit the federated update mechanism. Key differences:

• Goal: Gradient leakage aims to exfiltrate data; a backdoor aims to compromise model logic.
• Actor: Gradient leakage is often a server-side threat; backdoors are typically planted by malicious clients.
• Defense: Techniques like Secure Aggregation and Differential Privacy can mitigate gradient leakage but may not prevent a determined adversary from planting a backdoor.

Secure Aggregation is a cryptographic protocol that allows a federated learning server to compute the sum (or average) of client model updates without being able to inspect any individual client's contribution. It is a primary defense for client data privacy but has a nuanced relationship with backdoor defense:

• Privacy vs. Security: Secure Aggregation hides individual updates, protecting privacy but potentially hiding malicious backdoor updates within the crowd.
• Detection Challenge: The server cannot audit individual updates for backdoor signatures when Secure Aggregation is used.
• Complementary Use: It is often paired with Differential Privacy (which adds noise to the aggregate) to provide both privacy and some robustness against model manipulation.

Differential Privacy (DP) is a rigorous mathematical framework that bounds the influence any single data point can have on a computation's output. In federated learning, DP noise is typically added to model updates or the aggregated global model. Its role in mitigating Backdoor Attacks is indirect but significant:

• Mechanism: Adding calibrated noise (e.g., Gaussian) to the aggregated model can obfuscate the small, targeted weight perturbations that constitute a backdoor.
• Trade-off: The noise required to provide strong DP guarantees often degrades model utility (the privacy-accuracy trade-off), which an attacker might exploit by making the backdoor more resilient.
• Standard Practice: DP-SGD and its federated variants are considered a baseline defense against both privacy leaks and certain poisoning attacks.

Federated Averaging is the foundational algorithm for federated learning, where the server computes a weighted average of client model updates. Its simplicity is also its vulnerability to Backdoor Attacks:

• Attack Surface: The averaging operation is linear. A malicious client can scale its poisoned update to outweigh benign updates, ensuring the backdoor is incorporated into the global model.
• Persistence: Due to the averaging nature, a backdoor implanted in one round can be diluted by subsequent benign updates, requiring attackers to participate persistently.
• Mitigation: Advanced aggregation rules like FedProx (which adds a proximal term to limit client drift) or anomaly detection on update norms can be layered atop FedAvg to increase resilience.