Fine-Tuning Robustness

The watermark signal must survive the gradient-based optimization that occurs during fine-tuning. This requires the embedded pattern to be entangled with high-importance parameters that the fine-tuning process cannot alter without catastrophic forgetting of the primary task.

• Watermark is embedded in weights with large Fisher information values
• Fine-tuning naturally preserves these critical parameters
• Removal attempts cause significant accuracy degradation on the original task

When a model is adapted from one domain to another—such as from general image classification to medical imaging—the watermark must remain statistically detectable despite the distribution shift in both the model's internal representations and its output behavior.

• Trigger set samples must be domain-agnostic or abstract patterns
• Watermark key should correlate with features invariant to domain change
• Statistical watermarks in weight distributions often survive better than output-triggered methods

In scenarios where a base model is distributed to multiple licensees who each perform independent fine-tuning, the watermark must remain uniquely verifiable across all derivative models. This prevents collusion attacks where adversaries compare differently fine-tuned copies.

• Each licensee receives a uniquely watermarked copy
• Watermark survives independent adaptation paths
• Enables tracing of unauthorized redistribution back to the specific licensee

Modern fine-tuning often uses LoRA (Low-Rank Adaptation) or adapter layers that freeze the original weights and only train small auxiliary matrices. A watermark embedded in the frozen base weights remains intact, but the verification protocol must account for the added adapter parameters.

• Watermark in frozen base weights is naturally preserved
• Adapter layers do not overwrite the embedded signature
• Extraction must function with or without adapter weights present

The watermark embedding process itself must not create interference patterns that cause the model to forget previously learned knowledge when fine-tuned. This requires the watermark to be orthogonal to the model's natural learning dynamics.

• Embedding is performed during initial training, not post-hoc
• Watermark loss term is balanced against primary task loss
• Regularization ensures watermark does not dominate gradient updates during subsequent fine-tuning

After fine-tuning, the watermark detection must still produce a statistically significant result with a controlled false positive rate. The verification threshold must account for the noise introduced by the adaptation process.

• Null hypothesis: model is not watermarked
• Detection requires p-value below a predetermined threshold (e.g., p < 0.01)
• Correlation coefficient between secret key and model parameters remains above the decision boundary