Spectral Signatures

The detection pipeline begins by computing the spectrum of the feature covariance matrix for each class. For a class with N examples and d-dimensional feature vectors, the algorithm constructs a matrix M of size N×d and performs SVD. Poisoned samples exhibit a strong correlation with the top singular vector of the outlier class, creating a detectable separation in the right singular vector space. The spectral signature is the statistically significant projection of corrupted inputs onto directions of unexpectedly high variance.

Each training example is assigned an outlier score based on its correlation with the top right singular vector. The process:

• Compute the empirical distribution of correlation values for clean data
• Flag samples whose correlation exceeds a threshold defined by a robust z-score or median absolute deviation
• Apply iterative removal: after excising the top outliers, recompute the SVD to detect subtler poisoning This self-calibrating approach adapts to varying poisoning budgets without requiring prior knowledge of the trigger pattern.

Spectral signatures rely on a penultimate layer representation from a neural network that has been trained on the poisoned dataset. Key prerequisites:

• The network must have partially learned the backdoor association, meaning the trigger causes a consistent shift in the feature space
• The clean data for the target class must form a relatively compact cluster; diffuse clean distributions reduce detection power
• The poisoning ratio must be sufficient to create a statistically anomalous direction—typically above 3-5% of the target class Early-stage detection is possible by monitoring the singular value gap during training.

While both methods analyze internal representations, spectral signatures differ from activation clustering in key ways:

• Spectral signatures operate on the full feature matrix using linear algebra, avoiding the need for k-means or dimensionality reduction like PCA
• Activation clustering separates samples into two clusters per class and requires a heuristic to determine which cluster is poisoned
• Spectral signatures provide a continuous ranking of suspicion rather than a binary split, enabling more granular remediation
• The SVD approach is computationally cheaper for large datasets, scaling as O(Nd²) for the covariance computation.

Adversaries can design attacks to evade spectral signature detection:

• Distributed triggers: using multiple diverse trigger patterns dilutes the correlation with any single singular vector
• Feature-space blending: constraining poisoned samples to lie near the clean data manifold in feature space reduces outlier scores
• Low poisoning rates: below 2-3% of the target class, the anomalous signal may fall below the detection threshold
• Label-consistent attacks: clean-label poisoning where the trigger is embedded in correctly labeled examples can reduce the spectral anomaly Defenders should combine spectral signatures with gradient-based detection and data provenance checks for layered security.

A production-grade spectral signature defense follows this workflow:

1. Train a base model on the full dataset until validation accuracy plateaus
2. Extract feature vectors from the penultimate layer for all training examples
3. Group by label and compute the SVD of each class's feature matrix
4. Calculate outlier scores as the absolute correlation with the top right singular vector
5. Apply a threshold using median absolute deviation (MAD) with a multiplier of 6-10
6. Remove flagged samples and retrain the model from scratch on the sanitized dataset
7. Validate that the retrained model shows no backdoor behavior using a held-out trigger set