Model Distillation Resistance

The technique of adding statistical noise directly to a model's predictions or confidence scores to obscure the precise decision boundary from an attacker. By injecting calibrated noise, the teacher model's outputs become a lossy and unreliable source of supervision for a student model.

• Laplacian Noise: Adding noise drawn from a Laplace distribution to satisfy differential privacy guarantees.
• Gaussian Noise: Applying Gaussian noise to smooth out the confidence scores, making the decision surface harder to approximate.
• Trade-off: A critical balance exists between the magnitude of noise (defensive strength) and the degradation of service for legitimate users.

Limiting the number of output classes or top-k predictions returned by an API to reduce the information leakage per query. Instead of returning a full probability distribution over all classes, the API returns only the top-1 or top-5 labels.

• Top-1 Labeling: Returning only the final class label without any confidence score, providing minimal information for distillation loss functions.
• Gradient Starvation: A student model relies on rich soft labels to mimic the teacher's generalization. Truncation starves the student of these inter-class similarity signals.
• Implementation: Easily deployed at the API gateway without modifying the underlying model architecture.

Capping the amount of new information an attacker can derive from a single query, often measured by mutual information or entropy reduction. This technique monitors the sequence of queries from a session and blocks or degrades responses when the cumulative information gain exceeds a defined threshold.

• Entropy Thresholding: Blocking queries where the model's prediction entropy is high, as these boundary-probing queries are most valuable for mapping the decision surface.
• Adaptive Thresholds: Dynamically adjusting the allowed information gain based on the client's historical behavior and trust score.
• Mutual Information: A mathematical measure of the reduction in uncertainty about the model parameters given the query response.

Using a diverse ensemble of models to serve predictions, making the aggregate decision function inconsistent and harder to steal than a single model. A query is routed to a randomly selected model from the pool, so the attacker never interacts with a static function.

• Model Diversity: Ensembles built with different architectures, initializations, or training data splits create non-overlapping decision boundaries.
• Inconsistent Gradients: The student model's distillation loss fails to converge because the teacher's output distribution shifts unpredictably between queries.
• Operational Cost: Requires maintaining multiple models in production, increasing infrastructure complexity but providing a robust defense against extraction.

Monitoring API query sequences to detect the systematic, non-random access patterns indicative of an ongoing model extraction attack. This defense operates at the API gateway layer, analyzing metadata rather than model internals.

• Sequential Query Detection: Identifying extraction attempts by analyzing the temporal and spatial correlation between consecutive queries designed to map the input space.
• Session Fingerprinting: Building a unique profile of a client's querying behavior and device characteristics to link anonymous sessions and detect coordinated extraction campaigns.
• Response: Upon detection, the system can trigger rate limiting, serve decoy outputs, or inject response delays to disrupt the attack.

Introducing controlled randomness into the model's output logic so that identical queries do not always return the exact same result. This breaks the fundamental assumption of a deterministic teacher that distillation relies upon.

• Stochastic Inference: Enabling dropout layers or other stochastic elements during inference, not just training.
• Non-Deterministic Mapping: The student model cannot learn a stable function approximation because the target is a moving distribution.
• Consistency vs. Security: Legitimate applications requiring deterministic outputs must be whitelisted, as this defense intentionally sacrifices output consistency for security.

The technique of adding statistical noise directly to a model's predictions or confidence scores to obscure the precise decision boundary. By injecting calibrated noise, the defender prevents an attacker from accurately reconstructing the model's internal logic through querying. This is a direct implementation of differential privacy principles at the inference layer, trading a negligible drop in utility for a significant increase in extraction cost.

A training-time defense that modifies the model's loss function to create smoother or intentionally complex decision boundaries. Standard models often have sharp, easily approximated boundaries. Hardening forces the model to learn a mapping that is difficult for a surrogate model to replicate with limited queries. This often involves adversarial training regimes that specifically penalize boundary leakage.

A strategy that caps the amount of new information an attacker can derive from a single query. This is often measured by mutual information or entropy reduction. Techniques include:

• Prediction Truncation: Returning only the top-1 label instead of full probability vectors.
• Confidence Score Masking: Rounding or hiding raw confidence scores.
• Entropy Thresholding: Blocking queries where the model's prediction entropy is high, as these boundary-probing queries are most valuable for extraction.

A detection-based defense that monitors API query sequences to identify the systematic, non-random access patterns indicative of an ongoing extraction attack. Unlike simple rate limiting, this analyzes the semantic relationship between queries. Sequential Query Detection identifies attackers who methodically sweep the input space, while Session Fingerprinting links anonymous sessions to detect coordinated, distributed extraction campaigns.

A serving architecture that uses a diverse ensemble of models to answer queries, making the aggregate decision function inconsistent and harder to steal. By randomly routing queries to different sub-models or aggregating predictions in a non-linear fashion, the attacker cannot converge on a single, stable surrogate model. This exploits the fact that extraction attacks assume a static, deterministic target function.

A post-hoc verification technique that embeds a unique, verifiable identifier into a model's weights or behavior. This does not prevent extraction but provides a forensic mechanism to prove intellectual property ownership if a stolen model is deployed. Watermarks are typically triggered by a specific set of proprietary trigger inputs that produce a unique, pre-defined output, acting as a digital fingerprint for the model.