PATE Framework

The mathematical foundation upon which PATE is built. Differential privacy provides a formal guarantee that the output of a computation reveals no information about any single individual's record. In PATE, this is achieved by adding calibrated Laplacian or Gaussian noise to the teacher ensemble's vote counts before they are revealed to the student model. The privacy loss is quantified by the parameter epsilon (ε) — lower values indicate stronger privacy but reduced utility.

A primary threat vector that PATE is designed to defend against. In this attack, an adversary queries a trained model and analyzes its confidence scores to determine whether a specific record was present in the training data. PATE mitigates this risk because the student model never accesses the raw sensitive data — it only learns from noisy aggregated votes from teachers trained on disjoint partitions, making membership inference statistically infeasible.

The fundamental tension that PATE navigates. As more noise is added to teacher votes (stronger privacy), the student model's accuracy on real-world tasks degrades. Key factors influencing this balance include:

• Number of teachers: More teachers improve privacy but increase noise variance
• Noise scale: Higher noise protects privacy but reduces label quality
• Data partitioning strategy: Disjoint partitions prevent leakage but limit per-teacher data volume
• Student architecture: A well-regularized student can generalize better from noisy labels

The core machine learning technique underlying PATE's teacher-student architecture. Knowledge distillation transfers the decision boundaries learned by a complex teacher ensemble to a simpler student model by training the student on the teachers' aggregate outputs rather than ground-truth labels. In standard distillation, this compresses models; in PATE, it serves a privacy function — the student inherits the ensemble's collective knowledge without ever touching the sensitive training data.

A transparency artifact that should accompany any model trained via PATE. A model card documents:

• Training methodology: PATE with specified ε and teacher count
• Data provenance: Source and partitioning of sensitive data across teachers
• Privacy guarantees: Formal ε, δ bounds and noise mechanisms used
• Intended use: Tasks for which the student model is validated
• Limitations: Known failure modes and out-of-distribution behavior This documentation is essential for compliance with the EU AI Act's transparency requirements.

A GDPR principle that PATE operationalizes technically. Data minimization requires that personal data processing be adequate, relevant, and limited to what is necessary. PATE enforces this by design: the student model never ingests raw data; it only receives aggregated, noisy labels. The sensitive data remains confined to isolated teacher models, which can be discarded after training. This architectural constraint transforms a legal requirement into a verifiable technical property.