Defensive Distillation

Instead of training on hard, one-hot encoded labels (e.g., [0, 1, 0]), the student model is trained on the softmax output probability vectors of a pre-trained teacher model. These soft labels encode the relative similarity between classes, providing richer information about the decision boundaries. This forces the student to learn a smoother, more generalized function.

The primary defense mechanism is a form of gradient obfuscation. By smoothing the model's loss surface, the magnitude of the input gradients becomes extremely small in the vicinity of training points. Adversarial attacks like FGSM and PGD rely on these gradients to craft perturbations; distillation effectively starves them of directional information, causing attacks to fail.

A critical hyperparameter T (temperature) is introduced into the final softmax layer during training: softmax(z/T). A higher T produces a softer probability distribution, amplifying the knowledge about secondary classes. At inference time, T is typically set back to 1, preserving the smoothed decision logic while restoring confident predictions.

Defensive distillation is not tied to a specific neural network topology. It can be applied to any deep learning model that uses a softmax layer for classification. The technique has been successfully demonstrated on Deep Neural Networks (DNNs) and Convolutional Neural Networks (CNNs) for image classification tasks like MNIST and CIFAR-10.

While effective against white-box gradient-based attacks, distillation is vulnerable to black-box attacks and transferability. An attacker can train a substitute model on synthetically labeled data, craft adversarial examples against that proxy, and transfer them to the distilled model. It also offers limited protection against optimization-based attacks like the Carlini & Wagner (C&W) attack.

The defense requires a two-phase training pipeline. First, an initial teacher model must be trained to convergence on the original dataset. Second, the student model is trained using the teacher's soft labels. This doubles the training time compared to standard training, though the student model can often be architecturally identical to the teacher.

An input to a machine learning model that has been intentionally perturbed in a way imperceptible to humans, causing the model to make an incorrect classification with high confidence. Defensive distillation was specifically designed to mitigate the impact of these crafted inputs by smoothing the model's decision landscape.

A defensive technique that augments the training dataset with adversarial examples generated against the current model state. Unlike defensive distillation, which modifies the training objective using soft labels, adversarial training directly exposes the model to attacks, forcing it to learn a more robust decision boundary.

A phenomenon where a defense appears robust but actually relies on obfuscating or breaking the attacker's gradient signal. Defensive distillation was initially misinterpreted as a form of gradient masking; later research showed that while it can cause gradient vanishing, iterative white-box attacks like C&W can still bypass it.

An optimization-based adversarial attack that formulates the generation of adversarial examples as a constrained minimization problem. The C&W attack was pivotal in demonstrating that defensive distillation is not a complete solution, as it successfully defeated distilled networks by using a margin-based loss function to find minimally distorted perturbations.

The process of training a student model on the probability vectors (soft labels) of a teacher model. Defensive distillation applies this technique using the same architecture for both teacher and student, transferring knowledge about class similarities to reduce the model's sensitivity to small input variations.

The process of formally proving that a model's prediction is invariant to any perturbation within a defined Lp-norm ball. While defensive distillation provides empirical robustness, certification offers mathematical guarantees. Techniques like randomized smoothing provide probabilistic certificates that distillation alone cannot.