Model Extraction Attack

Targets models with simple, known architectures (e.g., logistic regression, shallow neural nets). The attacker queries the model to solve for the exact weights and biases analytically.

• Mechanism: Treats the model as a system of equations; n queries can solve for n unknown parameters.
• Critical Defense: Architecture secrecy is paramount, as knowledge of the structure enables direct mathematical theft.
• Example: Extracting a linear regression model's coefficients by querying d+1 strategically chosen data points.

The most common black-box attack. The adversary builds a functionally equivalent copy by training a new model on a synthetic dataset labeled by the target API.

• Process: 1) Generate queries (random, hill-climbing, or GAN-based). 2) Collect hard labels or soft labels (confidence scores). 3) Train a surrogate model to mimic the input-output mapping.
• Key Insight: Soft labels leak significantly more information per query, accelerating extraction.
• Example: Using a public dataset of faces to query a proprietary facial recognition API and training a local ResNet to replicate its embeddings.

Exploits the model's internal error states, edge cases, or confidence anomalies to infer decision boundaries without exhaustive querying.

• Method: Probes for adversarial examples or inputs that cause high prediction entropy to map the exact location of class boundaries.
• Efficiency: Requires far fewer queries than brute-force surrogate training by focusing on the most informative regions of the input space.
• Defense Note: Entropy thresholding and confidence score masking are direct countermeasures to this high-efficiency vector.

A physical or architectural attack that infers model architecture and weights by observing non-functional signals during inference, rather than just the output.

• Signals Exploited: GPU memory access patterns, inference latency, power consumption, or electromagnetic emanations.
• Target: Typically deployed against edge devices or shared cloud hardware where the attacker has co-location access.
• Example: Measuring the precise timing of matrix multiplications to deduce the layer dimensions of a model running on a compromised cloud instance.

Applies principles from cryptanalysis to reverse-engineer a model by treating it as an unknown cipher. Focuses on finding input-output collisions or invariants.

• Technique: Identifies functionally equivalent inputs—disparate inputs that produce identical outputs—to map the model's internal logic without knowing its parameters.
• Goal: Reconstruct the model's computational graph and activation functions, not just its final predictions.
• Defense: Feature space distortion and response randomization break the deterministic mapping this attack relies on.

The foundational defense against automated extraction. Rate limiting caps the total number of requests per time window, while query throttling intentionally slows response times for high-frequency clients.

• Disrupts the economic viability of extraction by increasing time-to-steal
• Can be implemented per-IP, per-session, or per-authenticated user
• Adaptive throttling uses query pattern analysis to apply stricter limits to suspicious sequences
• Often combined with proof-of-work challenges for high-risk sessions

Techniques that degrade the quality of information an attacker receives per query. Output perturbation adds calibrated statistical noise to predictions, while confidence score masking returns only the final class label instead of raw probability vectors.

• Differential privacy provides mathematical guarantees on information leakage
• Prediction truncation limits responses to top-k classes only
• Response randomization ensures identical queries don't always return identical results
• Obscures the precise decision boundary that surrogate models attempt to approximate

Monitoring systems that identify extraction attacks by analyzing the structure and sequence of incoming API queries. Extraction attempts exhibit systematic, non-random access patterns distinct from legitimate user traffic.

• Sequential query detection identifies grid-like sweeps of the input space
• Entropy thresholding flags boundary-probing queries with high prediction uncertainty
• Session fingerprinting links anonymous sessions to detect coordinated campaigns
• Triggers automated responses including blocking, decoy outputs, or alerting

Architectural defenses that make the model's decision function inherently difficult to approximate. Ensemble obfuscation routes queries through multiple models, creating an inconsistent aggregate function.

• Decision boundary hardening trains models with smoother, more complex boundaries
• Feature space distortion applies secret non-linear transformations to inputs
• Gradient masking hides the true gradient, defeating gradient-based extraction
• A stolen surrogate trained on ensemble outputs will fail to generalize accurately

Active deception techniques that mislead and poison extraction attempts. Honeypot models are decoy deployments designed to attract attackers, enabling security teams to study their methods.

• Decoy outputs serve deliberately incorrect predictions to high-risk clients
• Poisons the training data of any surrogate model being built by the attacker
• Query fuzzing injects imperceptible input variations to degrade surrogate quality
• Provides early warning of extraction campaigns before production models are targeted

Post-hoc intellectual property protection that embeds verifiable identifiers into model behavior. Model watermarking enables proof of ownership if a stolen model is deployed elsewhere.

• Watermarks can be embedded in weights, activation patterns, or specific input-output pairs
• Surrogate model detection compares suspicious models against proprietary trigger inputs
• Provides legal evidence for IP theft prosecution
• Complements active defenses by enabling enforcement after extraction succeeds