Pruning Resilience

Watermarks embedded in early layers are often more vulnerable to pruning, as these layers are heavily compressed during optimization. In contrast, signatures injected into the final classification layers or deeply entangled within bottleneck representations exhibit higher resilience. The key is to avoid placing the watermark signal in parameters that magnitude-based pruning algorithms will naturally identify as redundant.

Standard pruning algorithms target weights with the smallest absolute magnitudes. If a watermark is embedded by manipulating these low-magnitude parameters, it will be catastrophically erased. Resilient schemes, such as entangled watermarking, force the signature into high-magnitude, high-importance weights that are critical to the model's core functionality, making them immune to naive magnitude-based removal.

A model's overparameterization is a double-edged sword. While it provides ample capacity to hide a watermark without degrading fidelity, it also means a large percentage of weights can be pruned without harming accuracy. A watermark relying solely on this redundant capacity will be eliminated. True resilience requires the watermark to be distributed across a critical mass of parameters that the model cannot function without.

For black-box watermarks, the resilience of the trigger set is paramount. Simple, random noise triggers are easily forgotten after pruning. Robust triggers use adversarial perturbations or semantically meaningful samples with a high feature entanglement with the model's decision boundary. The more the trigger set resembles the distribution of the model's core training data, the more resistant it is to being pruned away.

The most effective defense is to anticipate the attack. Pruning-aware embedding simulates the pruning process during the watermark injection phase. This is often formulated as a min-max optimization problem:

• Inner loop: Simulates an attacker pruning the model to remove the watermark.
• Outer loop: Updates the watermark to survive the simulated attack. This proactive strategy forces the watermark to occupy a subspace of the weights that is robust to sparsification.

The type of pruning matters. Unstructured pruning, which zeros out individual weights, is less destructive to a well-entangled watermark than structured pruning, which removes entire neurons or channels. A watermark that survives 90% unstructured sparsity may be completely erased by 30% structured channel pruning. Resilience must be evaluated against the specific compression technique used in the target deployment environment.

The overarching property measuring a watermark's survival against deliberate erasure attempts. Pruning resilience is a specific subset of this broader category.

• Encompasses resistance to fine-tuning, compression, and distillation
• Evaluated by measuring Bit Error Rate (BER) after transformation
• A watermark with high robustness to removal maintains a false positive rate below 0.01% even after 90% of weights are pruned

A white-box method that embeds a signature by imposing a specific statistical bias on the distribution of the model's internal weights.

• Pruning resilience depends on embedding the key into high-magnitude weights that are less likely to be removed
• Verification uses correlation detection between the secret key and the parameter matrix
• Vulnerable to collusion attacks if multiple watermarked copies are compared

The specific ability of a watermark to survive transfer learning or domain adaptation. While distinct from pruning, the mechanisms of resilience often overlap.

• Both pruning and fine-tuning alter the model's weight space
• A watermark that survives magnitude-based pruning often resists fine-tuning on a new dataset
• Tested by applying a trigger set after the model has been updated on a downstream task

A removal technique where a student model is trained on the outputs of a watermarked teacher model. This process can wash away the watermark signal.

• Pruning resilience does not guarantee distillation resilience
• The student model learns the decision boundary but not the embedded weight patterns
• Defenses include designing trigger sets that produce high-entropy outputs the student is forced to replicate

The maximum amount of information, measured in bits, that can be reliably embedded and extracted. Pruning directly challenges capacity.

• Higher capacity often means lower pruning resilience, as more weights must be altered
• A payload embedding of a 64-bit user ID may degrade to a 0% Bit Error Rate (BER) after 80% sparsity
• Trade-off analysis is critical for model leasing applications requiring unique identifiers