Model Extraction

The attacker's goal is to train a clone model that approximates the decision boundary of the victim. This is achieved by:

• Sending carefully selected inputs to the public API
• Collecting the returned predictions (labels or confidence scores)
• Using this query-response dataset as a labeled training set
• Training a local model to mimic the victim's behavior The substitute model can then be used for adversarial example transfer, monetization, or to extract proprietary trade secrets embedded in the model's functionality.

For models that return full confidence vectors, extraction can be formulated as solving for the parameters of a known architecture. Key aspects:

• Logistic regression and shallow neural networks are particularly vulnerable
• Attackers can recover exact weights by solving systems of equations
• The number of queries required scales with the number of parameters
• Active learning strategies minimize query budgets by selecting maximally informative inputs This type of extraction achieves high fidelity with mathematical precision rather than approximation.

When only hard labels are returned, attackers reconstruct the decision boundary through systematic probing:

• Line-search attacks find the exact boundary between classes by querying along a direction
• Random walks along the boundary reveal its local geometry
• The collected boundary points serve as training data for the substitute
• This method requires more queries but works against minimal-output APIs
• Defenses like prediction throttling and confidence reduction aim to slow this process

A Knockoff Net is a substitute model trained on unlabeled public data labeled by the victim API. The attack exploits:

• The victim model acting as a high-quality annotator for natural inputs
• Transferability of adversarial examples between functionally similar models
• No need to know the victim's architecture or training data
• Effectiveness even against complex deep neural networks This approach demonstrates that query access alone is sufficient to steal model functionality at scale.

Attackers operate under practical constraints that shape extraction strategies:

• Query budgets may be limited by API rate limiting or cost
• Active sampling selects inputs near the decision boundary for maximum information gain
• Synthetic data generation creates queries when natural data is unavailable
• Transfer from public datasets reduces the number of API calls needed
• State-of-the-art attacks can achieve over 90% fidelity with fewer than 0.1% of the queries a naive approach would require

Extraction success is measured by how closely the substitute replicates the victim:

• Agreement rate: percentage of inputs where both models produce identical predictions
• Decision boundary distance: geometric similarity of classification surfaces
• Adversarial transfer rate: proportion of adversarial examples that fool both models
• Functional equivalence: whether the models behave identically on all possible inputs High fidelity enables downstream attacks and confirms successful intellectual property theft.

A formal characterization of an adversary's capabilities, goals, and knowledge, defining the assumptions under which a system's security is evaluated. Model extraction defenses must be designed against a specific threat model to be meaningful.

• Black-box access: Adversary can only query the API and observe input-output pairs — the most common extraction scenario
• Query budget: The maximum number of queries an attacker can make before economic or rate-limiting constraints apply
• Substitute architecture knowledge: Whether the attacker knows the victim model's architecture (white-box extraction) or must guess (black-box extraction)
• Label-only vs. confidence access: Whether the API returns full probability vectors or only the predicted class — dramatically impacts extraction difficulty