Model Extraction

The attacker sends a stream of carefully selected inputs to the victim model via a public API and collects the corresponding predictions. These input-output pairs form a labeled dataset used to train a substitute model that mimics the original's decision boundary. The fidelity of the clone depends on query budget, input diversity, and whether the victim returns confidence scores or only hard labels.

For models with known architectures and differentiable loss functions, extraction becomes a form of equation solving. The adversary crafts queries to numerically approximate the victim model's internal parameters by solving for the weights that minimize the disagreement between the substitute and the victim on strategically chosen inputs. This is particularly effective against logistic regression and shallow neural networks.

Advanced extraction attacks use active learning and adversarial example generation to synthesize inputs that lie near the victim model's decision boundary. These boundary-probing queries expose the most information per query, maximizing the fidelity of the stolen model while minimizing the query budget. Techniques include Jacobian-based dataset augmentation and query synthesis via generative adversarial networks.

Extraction is not limited to direct model outputs. Attackers exploit side channels such as response latency, memory access patterns, or floating-point timing variations to infer architectural details. Even when only top-1 labels are returned, timing analysis can reveal the depth of the computation graph or the number of classes, accelerating the construction of a functionally equivalent substitute.

The extracted model enables model piracy—the attacker can deploy the stolen functionality without incurring the original training costs, which may have involved millions of dollars in compute and proprietary data. The substitute can also serve as a white-box proxy for crafting adversarial examples that transfer to the victim model, enabling downstream evasion attacks.

Primary defenses include query rate limiting, prediction rounding to reduce information leakage, and injecting differential privacy noise into outputs. More sophisticated defenses involve detecting anomalous query patterns via out-of-distribution detection and returning deliberately misleading outputs when extraction is suspected. However, these defenses must balance security against degrading service quality for legitimate users.