Inferensys

Glossary

Threat Model

A formal characterization of an adversary's goals, knowledge, and capabilities, defining the specific security guarantees a defense must provide.
Knowledge manager reviewing enterprise knowledge management system on laptop, document library visible, casual office.
ADVERSARIAL SECURITY FRAMEWORK

What is a Threat Model?

A formal characterization of an adversary's goals, knowledge, and capabilities, defining the specific security guarantees a defense must provide.

A threat model is a structured representation of the security context for a machine learning system, formally defining the adversary's objectives, knowledge (white-box or black-box access), and capabilities (e.g., perturbation budget). It establishes the precise boundaries within which a defense must operate, moving security analysis from vague risk to quantifiable guarantees.

In modulation classification, a rigorous threat model specifies whether an attacker can manipulate raw IQ samples over-the-air, their maximum L_p-norm adversarial budget, and if they possess full gradient access to the classifier. This formalization is critical for evaluating certified robustness and distinguishing between theoretical vulnerabilities and practical, real-world evasion attacks.

ADVERSARIAL ROBUSTNESS

Core Components of a Threat Model

A formal threat model defines the rules of engagement for security analysis. It precisely characterizes the adversary's goals, knowledge, and capabilities, establishing the specific guarantees a defense must provide against evasion attacks on modulation classifiers.

01

Adversarial Goal

The specific security violation the attacker aims to achieve. In signal classification, this is typically an integrity violation—causing a targeted or untargeted misclassification of a modulation scheme.

  • Targeted Attack: The adversary forces the classifier to output a specific incorrect label (e.g., making BPSK appear as QPSK).
  • Untargeted Attack: The adversary simply wants any output other than the correct class.
  • Confidence Reduction: The attacker degrades the model's prediction confidence without changing the top-1 label, potentially triggering downstream uncertainty protocols.
02

Adversary Knowledge (White/Gray/Black Box)

The level of access and information the attacker possesses about the target model, which dictates the attack strategy's sophistication.

  • White-Box Access: Full knowledge of model architecture, weights, gradients, and training data. Enables powerful gradient-based attacks like Projected Gradient Descent (PGD).
  • Black-Box Access: Only input-output query access. The adversary must rely on transferability from a surrogate model or estimate gradients via zeroth-order optimization.
  • Gray-Box Access: Partial knowledge, such as the feature extraction pipeline but not the classifier head, or knowledge of the defense mechanism but not the model parameters.
03

Adversarial Capability (Perturbation Budget)

The constraints limiting how much an adversary can modify the input signal, formalized by an Lp-norm bound on the perturbation magnitude.

  • L∞ (Max Norm): Limits the maximum change to any single IQ sample. Common in image domains and directly applicable to waveform samples.
  • L2 (Euclidean Norm): Constrains the total energy of the added perturbation across all samples.
  • L0 (Sparsity): Limits the number of IQ samples the adversary can modify, relevant for pulsed or burst communication signals.
  • Over-the-Air Constraints: The perturbation must survive physical channel effects (fading, multipath) and comply with spectral mask regulations.
04

Attack Surface & Input Domain

The specific stage in the signal processing pipeline where the adversary injects the perturbation, defining the physical realizability of the threat.

  • Baseband IQ Injection: The adversary modifies the raw in-phase and quadrature samples before the receiver's classifier. Assumes direct digital access.
  • Over-the-Air (OTA) Waveform: The perturbation is transmitted as a physical waveform through a real radio channel. Must account for channel impairment and synchronization.
  • Feature-Level Attack: The adversary modifies extracted features (e.g., cumulants, cyclostationary signatures) rather than raw samples, assuming the feature extractor is compromised.
05

Defense Objective & Security Guarantee

The formal property the defense mechanism must provably or empirically maintain under the defined threat model.

  • Certified Robustness: A mathematical guarantee that the classifier's prediction remains constant for any perturbation within a verified Lp-ball radius. Achieved via randomized smoothing or formal verification.
  • Empirical Robustness: Measured accuracy against a specific attack algorithm (e.g., PGD). Does not provide a guarantee against stronger future attacks.
  • Detection & Rejection: The defense does not correct the classification but flags adversarial inputs for human review or discarding, using adversarial detection or out-of-distribution detection techniques.
06

Threat Model Mismatch & Adaptive Attacks

A critical failure mode where the defense is evaluated under a weaker threat model than a real adversary would use, leading to a false sense of security.

  • Gradient Obfuscation: A defense that relies on non-differentiable operations or shattered gradients to block gradient-based attacks, but fails against decision-based black-box attacks or Backward Pass Differentiable Approximation (BPDA).
  • Adaptive Adversary: An attacker who knows the defense mechanism and modifies their attack accordingly. Evaluating against a static attack without adapting to the defense invalidates the security claim.
  • Obfuscated Gradients: A specific sign of gradient masking where the model's loss surface provides no useful signal for first-order optimization, often easily circumvented.
THREAT MODEL FUNDAMENTALS

Frequently Asked Questions

A threat model formally defines the security assumptions, adversarial goals, and attack capabilities that a defense mechanism must withstand. Understanding these boundaries is essential for evaluating whether a modulation classification system provides meaningful protection or merely a false sense of security.

A threat model is a formal characterization of an adversary's goals, knowledge, and capabilities that defines the specific security guarantees a defense must provide. In the context of automatic modulation classification, the threat model specifies what the attacker knows about the classifier (white-box vs. black-box access), what perturbation budget they can expend (typically bounded by an Lp-norm constraint), and what constitutes a successful attack (targeted misclassification vs. untargeted evasion). Without a clearly defined threat model, robustness claims are meaningless—a defense that appears strong under one set of assumptions may be trivially broken under another. For example, a modulation classifier hardened against Fast Gradient Sign Method (FGSM) attacks may remain vulnerable to multi-step Projected Gradient Descent (PGD) adversaries operating under the same perturbation budget.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.