Query Pattern Analysis

Identifies extraction attempts by analyzing the temporal and spatial correlation between consecutive queries. Attackers mapping a decision boundary often query points in a structured, grid-like sequence.

• Mechanism: Monitors the vector distance between successive inputs.
• Indicator: A low, consistent delta between query features over a long session.
• Contrast: Genuine user traffic exhibits high entropy and random spacing in feature space.
• Action: Triggers a Session Fingerprinting alert and increases logging verbosity.

Blocks or flags queries where the model's prediction entropy is high. Boundary-probing queries, which are most valuable for extraction, naturally produce high uncertainty in the model's output distribution.

• Calculation: Shannon entropy over the softmax probability vector.
• Threshold: Flag queries exceeding the 95th percentile of baseline production entropy.
• Defense: Combine with Prediction Truncation to return only the top-1 label.
• Effect: Denies the attacker the rich confidence score gradients needed to train a surrogate model.

Caps the amount of new information an attacker can derive from a single query, often measured by mutual information or entropy reduction relative to previous responses.

• Strategy: Compare the current response distribution to the session's aggregate distribution.
• Defense: If the KL-divergence is too high, apply Output Perturbation to the response.
• Goal: Ensure no single query significantly shifts the attacker's understanding of the model.
• Implementation: Often deployed as a stateful proxy layer in front of the Secure Model Serving infrastructure.

Builds a unique profile of a client's querying behavior and device characteristics to link anonymous sessions and detect coordinated extraction campaigns.

• Passive Signals: TLS fingerprint (JA3/JA4), HTTP header order, canvas fingerprinting.
• Active Signals: API Tokenization and injected JavaScript challenges.
• Behavioral Analysis: Tracks typing cadence, query inter-arrival times, and feature-space traversal speed.
• Outcome: Correlates distributed, low-and-slow attacks that individually stay under rate limits.

The process of identifying unauthorized copies of a model by comparing their behavior on a set of proprietary trigger inputs to the original model's behavior.

• Watermarking: Embedding a Model Watermark that activates only on specific, out-of-distribution key inputs.
• Canary Queries: A secret set of inputs with known, deterministic outputs.
• Detection: Querying a suspected stolen API with canary inputs; a match constitutes proof of extraction.
• Legal Utility: Provides strong technical evidence for intellectual property litigation.

A decoy model deployed to attract attackers, allowing security teams to study extraction techniques and trigger alerts without exposing the production model.

• Design: A model with intentionally weakened Decision Boundary Hardening and no access to real data.
• Deception: Mimics the production API schema but returns subtly flawed or watermarked data.
• Intelligence: Captures the attacker's full query sequence to train future Query Pattern Analysis classifiers.
• Integration: Part of a broader AI Red Teaming Automation and active defense strategy.