Federated Coreset

The primary function of a federated coreset is to create a mathematical proxy for the entire decentralized dataset. It achieves this by selecting or synthesizing a small, weighted subset of points (or synthetic data) such that the loss function evaluated on the coreset closely approximates the loss evaluated on the full, distributed data. This enables the central server to make informed decisions—like guiding client selection or performing proxy updates—without direct access to client data. The quality of approximation is formally bounded by coreset guarantees.

Federated coresets are a privacy-enhancing technology. They operate on the principle of data minimization; instead of sharing raw gradients or model weights, clients may contribute only to the construction of a local coreset, or the server may infer a global coreset from aggregated metadata. This reduces the risk of model inversion or membership inference attacks. When combined with secure aggregation or differential privacy, coresets provide a strong, multi-layered privacy guarantee, making them suitable for highly regulated industries like healthcare and finance.

Coresets drastically reduce the communication overhead and computational burden in federated learning. By summarizing data into a compact form, the amount of information that needs to be transmitted from clients to the server is minimized. This is critical for bandwidth-constrained edge devices. Furthermore, operations performed on the small coreset (e.g., calculating gradients, estimating data importance) are far less computationally intensive than operations on the full dataset, enabling faster server-side processing and more efficient client selection algorithms.

A key application is guiding strategic client selection. The global coreset, representing the target data distribution, can be used to score or rank available clients. Selection strategies can prioritize clients whose local data distribution best matches or complements the coreset. For example:

• Select clients whose data minimizes the divergence from the coreset distribution.
• Use coreset importance weights to perform importance sampling for participant selection. This moves beyond random selection, accelerating convergence and improving model accuracy by ensuring each training round uses the most informative data.

Federated coresets are explicitly designed to address the challenge of Non-IID (Independent and Identically Distributed) data across clients. By constructing a coreset that approximates the global distribution, the system has a reference point against which local client skew can be measured and managed. This enables:

• Detection of clients with highly unusual or outlier distributions.
• Stratified sampling to ensure all data modes are represented.
• More robust global model updates that are less biased by any single client's local data skew. This is fundamental for building generalized models in real-world federated systems.

Coresets are not standalone; they integrate deeply with the federated optimization pipeline. They can be used to:

• Weight client updates during federated averaging based on how well a client's data aligns with the global coreset.
• Serve as a proxy dataset on the server for performing quick model updates or validation between communication rounds.
• Guide personalized federated learning by helping identify clusters of clients with similar data, enabling the creation of multiple, specialized coresets and models. This makes coresets a versatile component within broader federated learning orchestrators.

Client selection is the foundational process of determining which edge devices or data silos participate in a given training round. It directly impacts convergence speed, model fairness, and system efficiency. Strategies range from simple random sampling to complex, utility-based algorithms that consider data distribution, device resources, and historical performance. The goal is to select a cohort whose aggregated updates will most effectively improve the global model.

Importance sampling is a probabilistic client selection technique that assigns a higher probability of selection to clients deemed more "important." Importance is quantified by a utility metric, such as:

• The L2-norm of the client's gradient update, indicating the magnitude of local change.
• The size of the client's local dataset.
• The local training loss, signaling potential for improvement. This method biases selection toward clients whose contributions are expected to have a larger impact on global model convergence, making it a statistical precursor to coreset-based guidance.

Client profiling is the systematic collection and maintenance of metadata about federated learning participants. This metadata is the raw input for intelligent selection strategies, including coreset construction. Profiles typically track:

• Hardware capabilities (CPU, memory, NPU).
• Network conditions (bandwidth, latency).
• Data statistics (sample count, distribution features).
• Historical behavior (reliability, dropout rate, past contributions). A robust profiling system enables resource-aware, fair, and efficient client selection by providing the necessary context for decision-making algorithms.

Stratified sampling is a selection method that first divides the client population into non-overlapping subgroups (strata) based on key attributes. Attributes can include:

• Data distribution type (e.g., image classes per client).
• Geographic region.
• Device type or capability tier. Participants are then sampled from each stratum, ensuring the selected cohort is statistically representative of the overall population. This mitigates bias and improves model generalization, aligning with the distributional approximation goal of a federated coreset.

Resource-aware selection prioritizes clients based on their available system resources to enhance training efficiency and reliability. Key metrics considered are:

• Computational power and available memory.
• Battery level (for mobile/IoT devices).
• Network bandwidth and stability. By favoring well-resourced clients, this strategy reduces the incidence of stragglers—slow devices that delay round completion—and minimizes client dropout. It is often combined with statistical selection methods to balance efficiency with model quality.

Oort is an influential client selection framework that pioneered the joint optimization of statistical utility and system efficiency. It maintains client profiles and uses them to guide selection by:

1. Evaluating statistical utility via each client's training loss, prioritizing those with higher loss (greater potential to improve the model).
2. Incorporating system efficiency by considering time-to-completion, favoring clients with better resources. Oort's adaptive algorithm demonstrates how intelligent selection, informed by client metadata, can significantly accelerate federated learning convergence, a goal shared with coreset-guided methods.