Entropy Thresholding

The defense computes Shannon entropy over the model's softmax output vector: H = -Σ p(x_i) log p(x_i). A high entropy value indicates the model is uncertain—the query lies near a decision boundary where multiple classes have similar probabilities. These boundary queries are the most information-rich for an attacker building a surrogate model. Legitimate production traffic typically exhibits low entropy, with the model confidently assigning high probability to a single class. The threshold is calibrated based on the baseline entropy distribution of normal traffic.

Model extraction attacks rely on systematically probing decision boundaries to learn the shape of the model's function. Attackers craft queries that sit precisely between classes, where small input perturbations cause label flips. These queries produce maximum entropy because the model assigns near-equal probability to multiple classes. Entropy thresholding flags these queries by detecting the statistical signature of boundary exploration: sustained sequences of high-entropy predictions that deviate from the low-entropy pattern of genuine user traffic.

Static entropy thresholds are brittle—they either block legitimate ambiguous queries or allow sophisticated attackers to stay just below the cutoff. Adaptive calibration dynamically adjusts the threshold based on:

• Session context: A user's historical entropy baseline
• Query rate: Tightening thresholds as query frequency increases
• Global traffic patterns: Shifting the threshold in response to distributed attacks This Bayesian approach models the probability that a sequence of queries is malicious given their observed entropy values.

A critical implementation detail: entropy is not the inverse of confidence. A model can be highly confident (max softmax probability near 1.0) while still having high entropy if the remaining probability mass is spread across many classes. Conversely, a model with low max confidence can have low entropy if probability is concentrated in two classes. Effective thresholding operates on the full distribution, not just the top-1 confidence score. Attackers often exploit confidence-only defenses by crafting queries that produce moderate top-1 scores while extracting boundary information from the runner-up probabilities.

Entropy thresholding is most effective when layered with rate limiting. A single high-entropy query may be a legitimate edge case, but a sustained sequence of high-entropy queries from the same session is statistically anomalous. The defense tracks a cumulative entropy score over a sliding window. When the aggregate score exceeds a threshold, the system triggers graduated responses: first flagging, then throttling, and finally blocking the session. This temporal dimension prevents attackers from circumventing the defense by spacing out boundary queries.

In ensemble deployments, entropy thresholding can be strengthened by comparing prediction entropy across multiple models. A legitimate query typically produces consistent low-entropy predictions across all models in the ensemble. An extraction query probing a specific model's decision boundary may produce high entropy on the target model but not on others. This entropy divergence is a strong signal of targeted extraction. The defense can also route high-entropy queries to a hardened or watermarked model variant while serving low-entropy queries from the primary model.

The systematic monitoring of API query sequences to identify non-random, systematic access patterns indicative of extraction. While entropy thresholding evaluates the uncertainty of a single prediction, query pattern analysis examines the temporal and spatial correlation between consecutive queries. Attackers mapping a decision boundary will exhibit structured traversal of the input space rather than the random distribution of legitimate users.

• Detects grid-search and hill-climbing attack patterns
• Often combined with entropy metrics for layered defense
• Uses statistical divergence measures like KL-divergence

The practice of hiding or rounding raw confidence probabilities, often returning only the final class label. This directly complements entropy thresholding by removing the precise probability vector that entropy calculations require. Without access to the full softmax distribution, an attacker cannot compute which queries yield high-entropy boundary information.

• Return top-1 label only instead of full probability vector
• Round scores to coarse granularity (e.g., 0.1 increments)
• Eliminates the signal entropy thresholding monitors

Training models to have smoother or intentionally complex decision boundaries that resist surrogate approximation. Where entropy thresholding blocks queries that probe existing boundaries, boundary hardening proactively makes those boundaries harder to steal. Techniques include adversarial training with boundary-point samples and mixup regularization.

• Creates wider margin between classes
• Reduces the information value of any single boundary query
• Increases query cost for successful extraction

Capping the amount of new information an attacker can derive from a single query, measured by mutual information or entropy reduction. This is the theoretical foundation behind entropy thresholding: high-entropy predictions represent high information gain about the decision boundary. By blocking these queries, the defender directly limits the channel capacity of the extraction attack.

• Formalized through information-theoretic security models
• Complements rate limiting with per-query information budgets
• Measured in bits of information leaked per query

Introducing controlled randomness into model outputs so identical queries do not return identical results. This degrades the attacker's ability to use entropy measurements for boundary mapping by adding noise to the probability distribution. The defender can calibrate randomization to maintain accuracy for legitimate users while frustrating surrogate training.

• Apply calibrated noise to logits before softmax
• Use stochastic forward passes with dropout at inference
• Trade-off between utility preservation and extraction resistance

The process of identifying unauthorized copies by comparing behavior on proprietary trigger inputs against the original model. Entropy thresholding logs provide a forensic trail: queries that were blocked for high entropy often correspond to the exact boundary points a surrogate would need to learn. These blocked-query sets serve as canaries for detecting stolen models.

• Maintain a set of secret probing inputs with known outputs
• Compare suspected surrogate responses to original model
• Use blocked high-entropy queries as detection fingerprints