Local Model Accuracy

Local model accuracy is the primary performance metric for a machine learning model evaluated on the private, on-device dataset of a single client. It is calculated after the client performs local training or inference. This metric answers the question: "How well does the model work for this specific user or device?"

• Direct Measurement: Unlike global accuracy, it is measured directly on the client's raw data without requiring data to leave the device.
• Personalization Indicator: High local accuracy on a client's unique data distribution is the ultimate goal of personalized federated learning approaches.

A core characteristic of local model accuracy is its frequent divergence from global model accuracy. This divergence is primarily driven by statistical heterogeneity (non-IID data) across the federated network.

• Non-IID Data: If one client's data distribution (e.g., image types, word frequencies, sensor patterns) is significantly different from the population average, the globally aggregated model may perform poorly on that client's local data.
• Example: A global model trained on a federated network of smartphones might achieve 90% accuracy on a balanced test set. However, a client whose usage consists primarily of handwritten notes may see a local accuracy of only 65% for text recognition, while a client who mostly takes photos of documents may see 95% local accuracy.

The pursuit of high local model accuracy is the fundamental cause of client drift. During local training, each client's model parameters are updated to minimize loss on its local data distribution.

• Optimization Conflict: When clients have non-IID data, their local optimization steps pull the model parameters in different directions, away from the global optimum.
• Impact on Convergence: Excessive local training (many local epochs) maximizes local accuracy in the short term but can severely increase client drift, slowing or destabilizing global convergence. Techniques like FedProx add a proximal term to the local loss function to explicitly penalize deviation from the global model, managing this trade-off.

In personalized federated learning, the explicit goal is to maximize each client's local model accuracy, even at the expense of global model performance. The system produces a unique model for each client that is tailored to its data.

• Personalization Techniques: Methods include:
  • Local Fine-Tuning: Taking the global model and performing a few additional epochs of training solely on local data.
  • Model Interpolation: Learning a client-specific model as a weighted combination of the global model and a locally trained model.
  • Meta-Learning: Training a global model that is explicitly designed to be easily adaptable (few-shot) to new clients' local data.

Local model accuracy is not just a final metric; it is a fundamental data point for federated evaluation. The server can securely aggregate local accuracy scores from clients to estimate global performance without centralizing data.

• Secure Aggregation of Metrics: Clients compute accuracy locally and send only the scalar metric (e.g., 0.87) to the server. Cryptographic secure aggregation protocols can sum these metrics without revealing any individual client's value.
• Estimating Performance Distribution: By collecting many local accuracy scores, the server can understand the distribution of model performance across the network—identifying if performance is equitable or if certain client subgroups are poorly served.

The measurable local model accuracy is directly affected by the system heterogeneity of the federated network. Device limitations constrain how much local training can be performed, impacting final local performance.

• Compute/Memory Limits: A client with a powerful GPU can train for more local epochs, potentially achieving higher local accuracy than a client on a microcontroller.
• Straggler Effect: Clients that are slow to compute (stragglers) may be dropped from a training round, meaning their local data distribution and potential accuracy gains are not incorporated into the global model, potentially biasing it.
• On-Device Optimization: Techniques like quantization and pruning reduce model size for edge deployment but can introduce a small drop in local accuracy compared to the full-precision model.

The performance of the aggregated federated model on a held-out test set representative of the overall target population. This is the primary objective of federated learning, but it can differ significantly from local accuracy due to statistical heterogeneity.

• Key Insight: A high global accuracy does not guarantee high local accuracy for all clients, especially those with unique data distributions.

The phenomenon where local models, trained on non-IID client data, diverge from the global model objective. This divergence causes slow or unstable model convergence.

• Mechanism: Occurs because local optimization on biased data pulls model parameters away from the global optimum.
• Mitigation: Techniques like FedProx add a regularization term to local loss functions to constrain drift.

Measures how well a globally trained federated model can be adapted to perform accurately on a specific client's data distribution.

• Evaluation: After global training, a model is fine-tuned locally on a client's private data. Performance on the client's local validation set measures personalization success.
• Techniques: Includes meta-learning approaches (e.g., Per-FedAvg) and model interpolation between global and local models.

The decentralized process of assessing model performance across clients without centralizing private data. Metrics like accuracy, loss, and fairness metrics are computed locally and aggregated (e.g., averaged) on the server.

• Protocol: The server sends the global model to a subset of clients; they compute metrics on local test sets and send only the results back.
• Challenge: Ensuring the aggregated metric (e.g., mean accuracy) is statistically meaningful across heterogeneous clients.

The difference in performance between a model on its local training data and its performance on unseen global data. In federated learning, this gap is often wide due to non-IID data.

• Local Overfitting: A model may achieve high local accuracy by overfitting to its client's specific distribution, but fail on data from other clients.
• Measurement: Can be estimated via cross-client validation, where a model is validated on held-out data from other clients.

The inverse relationship where increasing formal privacy guarantees (e.g., via Differential Privacy (DP)) typically reduces the final model's accuracy or utility.

• Mechanism: Adding DP noise to client updates or aggregated models protects data but degrades the signal, impacting convergence rate and final global model accuracy.
• Parameter: The epsilon (ε) privacy budget directly controls this trade-off; lower ε means stronger privacy but potentially lower utility.