Neural Cleanse

Neural Cleanse assumes a backdoor exists and attempts to reconstruct the trigger by solving an optimization problem. For each possible target label, it searches for the minimal perturbation required to cause consistent misclassification.

• Starts with a small patch or mask pattern
• Iteratively modifies the trigger to minimize its size while maximizing misclassification
• Uses the L1-norm to enforce sparsity, ensuring the recovered trigger is compact
• The trigger with an anomalously small norm indicates a likely backdoor

After reconstructing candidate triggers for all labels, Neural Cleanse performs statistical outlier detection on the trigger norms. A genuine backdoor trigger requires significantly less modification than a benign label.

• Computes the Median Absolute Deviation (MAD) of all trigger norms
• Flags any label whose trigger norm exceeds a threshold as compromised
• This step distinguishes true backdoors from the natural minimum perturbation required to flip a clean model's predictions

Once a backdoor trigger is identified, Neural Cleanse patches the model to neutralize the vulnerability. The recovered trigger is used to generate a remediation dataset.

• The reverse-engineered trigger is stamped onto clean samples with the correct original label
• The model is fine-tuned on this augmented dataset for a small number of epochs
• This process overwrites the malicious association without degrading performance on clean inputs
• The patching is lightweight and does not require retraining from scratch

Neural Cleanse operates under specific constraints that define its threat model and practical applicability.

• Assumes the trigger is a static patch or pattern, not a complex transformation like image warping or filter-based triggers
• Requires white-box access to model weights and architecture for gradient-based optimization
• Computationally expensive for models with many output classes, as each label requires a separate reconstruction
• May produce false positives if a class naturally has a very small decision boundary
• Does not defend against source-agnostic backdoors that use dynamic or input-dependent triggers

A pre-training detection method that identifies poisoned examples before they corrupt the model. It operates on the principle that backdoor samples leave a detectable statistical trace in feature representations.

• Analyzes the singular value decomposition of hidden layer activations for each class
• Poisoned samples with a shared trigger tend to cluster along a dominant singular vector
• Computes an outlier score for each training example; those exceeding a threshold are flagged
• Complements Neural Cleanse by catching attacks at the data level rather than the model level

A post-detection mitigation technique that surgically removes backdoor neurons without retraining from scratch. Often paired with Neural Cleanse after a trigger is identified.

• Step 1: Record neuron activations on clean validation data to identify dormant neurons
• Step 2: Prune neurons that are consistently inactive on clean data but fire on trigger-embedded inputs
• Step 3: Fine-tune the pruned model on a small clean dataset to recover any lost accuracy
• Advantage: Computationally cheap compared to full retraining; preserves benign performance

A defense-by-detection method that separates clean and poisoned training data by analyzing internal model representations. It operates on the assumption that backdoor samples form a distinct cluster in activation space.

• Extracts the final hidden layer activations for all training samples within a class
• Applies dimensionality reduction (PCA or t-SNE) and k-means clustering (k=2)
• If one cluster is significantly smaller and exhibits consistent misclassification, it is flagged as poisoned
• Strength: Agnostic to trigger shape or size; works on semantic backdoors

A provable defense that constructs a classifier with mathematical guarantees against adversarial perturbations, including backdoor triggers. It transforms any base classifier into a certifiably robust one.

• Adds isotropic Gaussian noise to inputs during inference
• Returns the class that is most probable under the noise distribution
• Provides a certified radius: a guarantee that the prediction will not change for any perturbation smaller than that radius
• Limitation: The certified radius may be too small to cover large backdoor patches, but it offers formal verification that Neural Cleanse's empirical approach lacks

A model-level sanitization strategy that transfers benign knowledge from a potentially poisoned teacher model to a clean student model, aiming to discard backdoor behavior during the transfer.

• Teacher: The original, potentially backdoored model
• Student: A fresh model trained only on the soft labels (probability distributions) produced by the teacher on a clean dataset
• Hypothesis: Backdoor triggers cause sharp, high-confidence misclassifications that are smoothed out in soft labels
• Use case: When you cannot access or retrain the original training data, only a clean surrogate dataset