Weight Regularization

The watermark is embedded by augmenting the primary task loss function with a regularization term that penalizes deviations from a target weight distribution. This term is weighted by a hyperparameter lambda, which controls the trade-off between watermark strength and model fidelity. During backpropagation, the optimizer minimizes both task error and watermark constraint violation simultaneously.

• The regularization term typically enforces a Gaussian or binary statistical signature on selected layers
• Higher lambda values increase watermark detectability but risk accuracy degradation
• The loss landscape is shaped to create a local minimum that encodes the owner's identity

Unlike trigger-set methods, weight regularization directly encodes a statistical pattern into the parameter distribution. The owner defines a target mean, variance, or correlation structure for specific weight matrices. The regularization loss minimizes the divergence between the actual weight distribution and this target signature using metrics like Kullback-Leibler divergence or Maximum Mean Discrepancy.

• Signatures can be embedded in convolutional filters, attention heads, or fully-connected layers
• The pattern is mathematically improbable to occur through random initialization or standard training
• Extraction requires white-box access to compare weight statistics against the secret target distribution

A critical design constraint is that the watermark must not cause statistically significant degradation on the primary task. This is achieved through careful tuning of the regularization coefficient and selective application to redundant or over-parameterized layers. Modern approaches use gradient projection to ensure watermark updates are orthogonal to task-critical directions in weight space.

• Watermark updates are constrained to the null space of the task gradient
• Layers with high intrinsic dimensionality offer more capacity for covert embedding
• Validation loss is monitored throughout training to detect interference early

Weight regularization watermarks exhibit strong overwriting resistance because the signature is distributed across many parameters. An adversary attempting to embed a new watermark would need to overwrite the existing statistical pattern, which requires destructive fine-tuning that significantly degrades model utility. The original signature persists as a residual statistical artifact even after aggressive retraining.

• Distributed embedding across thousands of weights creates redundancy
• Partial overwriting leaves detectable fragments of the original signature
• The capacity of the weight space allows multiple signatures to coexist if properly orthogonalized

A key advantage of weight regularization watermarks is resilience to magnitude-based pruning and quantization. Since the signature is encoded in the statistical distribution rather than individual weight values, pruning low-magnitude weights does not erase the pattern. The watermark survives compression techniques like INT8 quantization because the relative statistical properties are preserved even at reduced precision.

• The signature is embedded in the collective behavior of weights, not isolated values
• Post-training quantization preserves distributional characteristics
• Structured pruning of entire channels or filters may degrade the watermark if applied to signature-bearing layers

Ownership verification requires the legitimate owner to present the secret target distribution and demonstrate that the suspect model's weights match this pattern with statistical significance. A null hypothesis test is performed: the probability that the observed weight statistics match the target by random chance must be below a threshold (typically p < 10⁻⁶). This provides a rigorous mathematical basis for legal IP claims.

• The detection key consists of the target distribution parameters and the layer indices
• False positive rate is analytically computable from the statistical test
• Third-party arbiters can verify claims without accessing training data