Decision Boundary Hardening

Augments the training dataset with adversarial examples—inputs perturbed to cause misclassification—to force the model to learn a smoother, more regularized decision boundary. This reduces the presence of sharp, easily exploitable cliffs in the loss landscape.

• Projected Gradient Descent (PGD): Iteratively generates strong adversarial examples during training.
• TRADES: Optimizes a trade-off between natural accuracy and robustness, directly smoothing the boundary.
• Result: A surrogate model trained on standard queries fails to capture the nuanced, hardened boundary.

Trains a model on convex combinations of input pairs and their corresponding labels, enforcing linear behavior between training samples. This acts as a strong regularizer that flattens the decision landscape.

• Standard Mixup: Interpolates raw inputs: x_new = λx_i + (1-λ)x_j.
• Manifold Mixup: Performs interpolation in the latent feature space, smoothing higher-level representations.
• Extraction Impact: The resulting boundary is highly ambiguous for off-manifold queries, yielding low-confidence predictions that starve a surrogate model of useful gradient information.

Directly penalizes the Frobenius norm of the model's input-output Jacobian matrix during training. This explicitly minimizes the sensitivity of predictions to infinitesimal changes in the input, creating a flat, low-curvature decision boundary.

• Mechanism: Adds a penalty term λ||∂y/∂x||² to the loss function.
• Effect: Forces the model to be locally constant, drastically reducing the information leakage per query.
• Key Benefit: A surrogate model cannot easily approximate a function whose gradient is intentionally suppressed.

Randomly drops or perturbs internal activations during inference, creating a non-deterministic decision function. An attacker querying the same input multiple times receives slightly different outputs, blurring the perceived boundary.

• Dropout at Inference: Keeps dropout layers active during deployment, not just training.
• Gaussian Noise Injection: Adds calibrated noise to hidden layer outputs.
• Extraction Defense: The surrogate model learns an averaged, softened version of the boundary, losing the sharp details necessary for high-fidelity theft.

Replaces hard one-hot encoded training targets with soft targets that assign a small probability mass to incorrect classes. This discourages the model from pushing decision boundaries into extreme, high-confidence regions.

• Implementation: y_smooth = (1 - ε) * y_one_hot + ε / K.
• Result: The model maintains a margin of uncertainty, preventing the sharp, overconfident boundaries that are easiest for surrogate models to map.
• Synergy: Combines effectively with knowledge distillation resistance to create a teacher model that is inherently difficult to steal.

Deploys an ensemble of models that are explicitly trained to disagree on off-manifold or boundary-probing inputs while agreeing on clean, in-distribution data. The aggregate prediction function becomes highly non-linear and inconsistent from the attacker's perspective.

• Diversity Loss: Adds a penalty term that maximizes the variance of ensemble member predictions on adversarial inputs.
• Adaptive Querying Frustration: An attacker's strategy of adaptive sampling fails because the effective boundary shifts unpredictably.
• Outcome: The surrogate model learns a crude, low-accuracy approximation of the true ensemble boundary.