Watermark Capacity

The inherent excess capacity of a neural network's weights is the primary reservoir for watermark data. Over-parameterized models possess many near-equivalent local minima, allowing a watermark to occupy a distinct, non-interfering subspace. Capacity scales with total parameter count, but the relationship is not linear. A model's intrinsic dimensionality—the minimum number of parameters needed to represent its function—defines the true upper bound. Parameters outside this manifold are redundant and can be statistically biased to encode a payload without shifting the decision boundary.

Embedding information is a constrained optimization problem. Every bit of payload injected into the model's weights or activations acts as a regularizer, pulling the model away from its optimal loss basin. The fidelity preservation constraint dictates that the watermark signal must remain below the noise floor of the model's generalization error. Key relationships include:

• Payload vs. Accuracy: A higher bit capacity requires a stronger statistical bias, which eventually degrades benchmark performance.
• Capacity vs. Robustness: A larger payload is often more fragile, as it occupies more parameter space susceptible to fine-tuning or pruning.

The extraction interface fundamentally limits capacity. White-box methods access millions of weight parameters directly, enabling high-capacity payload embedding (potentially hundreds of bits) by imposing a statistical structure detectable via correlation detection. Black-box methods are bottlenecked by the output layer. Capacity is limited to the number of distinct trigger set samples that can be reliably distinguished. A model with a 1000-class output can theoretically encode up to log2(1000) bits per trigger, but practical capacity is far lower due to the need for statistical significance in ownership verification.

To survive removal attacks, a watermark must be embedded in the most salient, task-critical features—a process known as entangled watermarking. This directly competes with capacity. Forcing the watermark to be robust against fine-tuning or pruning requires it to occupy the low-rank, high-energy subspaces of the weight matrices. This is prime real estate for the primary task, meaning a highly robust watermark consumes a disproportionate amount of the model's effective capacity, severely limiting the residual space available for a high-bit-rate payload.

Capacity is not just about how many bits are embedded, but how reliably they can be read back. The Bit Error Rate (BER) is the definitive metric. A scheme is said to have a capacity of N bits if the BER is below a threshold (e.g., < 1%) under expected distortions. The relationship is governed by information theory:

• Channel Capacity: The model is a noisy channel; its Shannon capacity defines the theoretical maximum error-free bit rate.
• Error Correction: Overhead from error-correcting codes reduces the net payload but is essential for achieving a legally defensible false positive rate.

An often-overlooked factor is the capacity required to resist collusion attacks. If an adversary obtains multiple differently watermarked copies, they can average the weights to isolate the common signal (the model) from the varying signals (the watermarks). To resist this, the embedding scheme must use a high-dimensional, orthogonal key space. This requires a larger capacity footprint, as the watermark must be spread across many parameters in a unique pattern for each user, reducing the per-user payload but enabling traitor tracing.