Prediction Vector

The prediction vector is a probability mass function where each element corresponds to a class label and sums to 1.0. The values represent the model's calibrated confidence that the input belongs to each class. In a standard softmax output, even incorrect classes receive non-zero probabilities, creating a rich information signal that can be exploited by model inversion attacks to reconstruct representative class features.

The full prediction vector leaks significantly more information than a hard label. The relative confidence margins between the top-k classes encode the model's decision boundary geometry. Attackers exploit this by performing gradient-based optimization on a dummy input to match the target prediction vector, effectively reversing the forward pass to reconstruct training data. This is the core mechanism behind Deep Leakage from Gradients (DLG) and confidence-based inversion.

A primary defense truncates the prediction vector to return only the top-k class labels without their associated probabilities. By suppressing the exact confidence values and zeroing out low-probability classes, the attack surface for inversion is dramatically reduced. This technique trades off some model interpretability for a substantial increase in privacy preservation, as the attacker loses the fine-grained gradient signal needed for reconstruction.

The softmax temperature parameter T controls the sharpness of the probability distribution. A high temperature (T > 1) produces a softer, more uniform distribution that masks the model's true confidence margins, acting as a natural defense. Conversely, a low temperature (T < 1) sharpens the distribution toward a one-hot vector, which can reduce information leakage but may also impact model calibration and training dynamics in defensive distillation.

In differential privacy frameworks, the prediction vector is a direct target for output perturbation. Calibrated Gaussian or Laplacian noise is added to the raw logits or softmax probabilities before release. The sensitivity of the prediction vector—how much it changes when a single training record is modified—determines the privacy budget (epsilon) consumed per query. This mechanism provides formal guarantees against inversion and membership inference.

Even when confidence scores are hidden, the hard-label prediction alone can be exploited. In label-only inversion attacks, an adversary queries the model with perturbed inputs and observes the decision boundary crossing points. By mapping the geometry of where the predicted class changes, the attacker can reconstruct class-level prototypes without ever seeing a prediction vector, demonstrating that defense strategies must consider the entire query-response interface.

A defense mechanism that truncates the prediction vector to return only the top-k class labels instead of full probability distributions. By hiding low-probability scores, the information available for inversion attacks is dramatically reduced.

• Top-1 masking: Returns only the predicted class label
• Top-5 masking: Returns the five highest-confidence classes
• Trade-off: Reduces model explainability and calibration visibility

A privacy mechanism that directly adds random noise to the prediction vector before returning it to the user. Unlike DP-SGD which protects training data, output perturbation protects individual inference queries.

• Noise calibration: Scaled to the sensitivity of the softmax function
• Laplace mechanism: Common choice for L1 sensitivity
• Gaussian mechanism: Preferred for L2 sensitivity bounds