Glossary
Federated Edge Learning
Federated Averaging Algorithms
Terms related to the core iterative algorithms for aggregating client updates in federated learning. Target: ML Engineers & Researchers.
Federated Averaging (FedAvg)
Federated Averaging (FedAvg) is the foundational iterative algorithm for federated learning where a central server coordinates multiple clients to train a shared global model by periodically averaging their locally computed model updates.
Local Epochs
Local epochs refer to the number of complete passes a federated learning client makes over its local dataset during a single communication round before sending an update to the server.
Communication Round
A communication round, also known as a federated round, is a single cycle in federated averaging where the server broadcasts the global model, selected clients perform local training, and the server aggregates their updates to form a new global model.
Client Participation Rate
The client participation rate is the fraction of the total available clients that are selected to participate in a given federated learning communication round.
Model Delta
A model delta is the difference between a client's locally updated model parameters and the global model parameters it received at the start of a federated learning round, which is transmitted to the server for aggregation.
Update Aggregation
Update aggregation is the server-side process in federated learning that combines the model deltas from participating clients, typically via a weighted average, to produce a new global model.
Synchronous Aggregation
Synchronous aggregation is a federated learning protocol where the server waits for updates from all selected clients in a round before proceeding to aggregate them and update the global model.
Asynchronous Aggregation
Asynchronous aggregation is a federated learning protocol where the server updates the global model immediately upon receiving an update from any client, without waiting for a full round to complete.
Client Drift
Client drift is the phenomenon in federated learning where local client models diverge from the global objective due to optimization on statistically heterogeneous (non-IID) local data, hindering convergence.
FedProx
FedProx is a federated optimization algorithm that modifies the local client objective by adding a proximal term to constrain local updates, mitigating client drift caused by data heterogeneity.
SCAFFOLD
SCAFFOLD (Stochastic Controlled Averaging for Federated Learning) is an algorithm that uses control variates to correct for client drift, achieving faster convergence under non-IID data by reducing the variance in client updates.
FedNova
FedNova (Federated Normalized Averaging) is a federated averaging variant that normalizes client updates based on their local optimization steps to account for heterogeneity in client workloads, improving convergence stability.
FedOpt Framework
The FedOpt framework is a generalized approach to federated optimization where the server applies adaptive optimizers like Adam or Yogi to the aggregated client updates, rather than simple averaging.
FedAdam
FedAdam is a federated optimization algorithm within the FedOpt framework where the server uses the Adam optimizer to adaptively update the global model based on aggregated client gradients.
FedAvgM
FedAvgM (Federated Averaging with Server Momentum) is an algorithm that incorporates a momentum term on the server side during the aggregation step to stabilize and accelerate the convergence of the global model.
Weighted Averaging
Weighted averaging is the standard aggregation method in federated averaging where client updates are combined proportionally to the size of their local datasets, ensuring the global objective aligns with the overall data distribution.
FedSGD
FedSGD (Federated Stochastic Gradient Descent) is a baseline federated algorithm where clients perform a single batch gradient descent step per round, equivalent to FedAvg with one local epoch and a full batch size.
Partial Client Participation
Partial client participation is a practical federated learning scenario where only a subset of the total client pool is available or selected for training in each communication round, due to system constraints like connectivity or resource limits.
Straggler Mitigation
Straggler mitigation refers to techniques in federated learning, such as asynchronous aggregation or deadline-based selection, designed to handle clients that are significantly slower than others, preventing them from bottlenecking the training process.
Update Compression
Update compression encompasses techniques like quantization and sparsification applied to model updates in federated learning to reduce their size, thereby decreasing the communication bandwidth required between clients and the server.
Gradient Clipping
Gradient clipping is a technique used in federated learning where the norm of client gradients or model updates is capped before transmission to improve training stability, provide differential privacy, and mitigate the impact of outliers.
FedPer
FedPer (Federated Learning with Personalization Layers) is a personalized federated learning approach where the base layers of a model are learned collaboratively, while the final layers (head) are personalized and kept local to each client.
FedRep
FedRep (Federated Representation Learning) is a personalized federated learning algorithm where clients collaboratively learn a shared data representation (body) while training unique local heads for personalized prediction.
FedBN
FedBN (Federated Batch Normalization) is a method where batch normalization statistics (mean and variance) are not aggregated but remain local to each client, improving performance on non-IID data by accounting for feature distribution shifts.
Global Model
The global model is the shared neural network in federated learning whose parameters are maintained and updated by the central server through the aggregation of client updates, representing the collaborative knowledge learned from all participants.
Local Model
A local model is an instance of the global model that is downloaded by a federated learning client, trained on the client's private local dataset for a number of epochs, and then used to generate an update for the server.
Statistical Heterogeneity
Statistical heterogeneity, or non-IID data, refers to the fundamental challenge in federated learning where the data distributions across clients are not independent and identically distributed, leading to convergence issues like client drift.
Convergence Guarantee
A convergence guarantee in federated learning is a formal mathematical proof that under specified conditions (e.g., convexity, bounded gradients), an algorithm like FedAvg will converge to a stationary point of the global objective function.
Federated Training Loop
The federated training loop is the core iterative process encompassing client selection, model broadcast, local training, update transmission, server-side aggregation, and global model update that defines the execution of federated averaging.
Client Selection Strategies
Terms related to the methods for choosing which edge devices participate in a federated learning round. Target: System Architects & CTOs.
Client Selection
Client selection is the process of determining which edge devices or data silos participate in a given round of federated learning training.
Random Selection
Random selection is a client selection strategy where participants for a federated learning round are chosen uniformly at random from the available pool.
Stratified Sampling
Stratified sampling is a client selection method that divides the client population into subgroups (strata) based on attributes like data distribution or device type and samples from each to ensure representative participation.
Power-of-Choice
Power-of-choice is a client selection heuristic that evaluates a small random subset of clients and selects the one with the highest utility, such as the largest local dataset or fastest connection, to improve convergence speed.
Importance Sampling
Importance sampling is a probabilistic client selection technique that weights the probability of selecting a client based on an importance metric, such as the norm of its model update or the size of its local dataset.
Resource-Aware Selection
Resource-aware selection is a client selection strategy that prioritizes devices based on their available computational power, memory, battery level, and network bandwidth to improve system efficiency and reduce stragglers.
Fairness-Aware Selection
Fairness-aware selection is a client selection approach that incorporates fairness constraints, such as ensuring devices or data distributions are not systematically underrepresented, to mitigate bias in the global model.
Incentive Mechanism
An incentive mechanism is a system design element, often economic or reputational, that encourages client devices to participate truthfully and reliably in federated learning by compensating them for their resources and data contributions.
Client Profiling
Client profiling is the process of collecting and maintaining metadata about federated learning clients, including their hardware capabilities, network conditions, data statistics, and historical behavior, to inform selection decisions.
Gradient Norm
Gradient norm is a metric used in client selection, where the magnitude (norm) of a client's local model update is calculated as a proxy for the significance or contribution of its update to the global model.
Shapley Value
The Shapley value is a concept from cooperative game theory used in federated learning to quantify the marginal contribution of each client's data to the performance of the global model, often for contribution valuation or selection.
FedCS
FedCS (Federated Learning with Client Selection) is a seminal framework and protocol designed to manage heterogeneous client resources by selecting participants based on their resource availability and deadlines to improve training efficiency.
Oort
Oort is a client selection framework that jointly optimizes for statistical utility (based on client training loss) and system efficiency (based on client resource profiles) to accelerate federated learning convergence.
TiFL
TiFL (Tier-based Federated Learning) is a client selection strategy that groups clients into tiers based on their training performance and selects participants from each tier in each round to handle system heterogeneity and improve convergence.
Straggler Mitigation
Straggler mitigation refers to techniques in client selection and system design that prevent or manage slow or unresponsive devices from delaying the completion of a federated learning training round.
Multi-Armed Bandit
In client selection, a multi-armed bandit is an online learning framework used to sequentially choose clients (arms) to maximize a reward (e.g., model improvement) while balancing exploration of new clients and exploitation of known high-performers.
Client Clustering
Client clustering is a pre-selection strategy that groups clients based on similarity in data distribution, device capability, or other features to enable more efficient cohort-based or stratified selection and training.
Cohort Selection
Cohort selection is a client selection method where groups of clients (cohorts) are chosen and trained together, often used in cross-silo settings or to manage privacy and scalability in large-scale deployments.
Client Scoring
Client scoring is the process of assigning a numerical value or rank to each potential participant based on a utility function that combines factors like data quality, resource availability, and historical contribution.
Utility Function
In client selection, a utility function is a mathematical formula that quantifies the expected benefit of selecting a specific client, typically balancing objectives like model accuracy, training speed, and fairness.
Bias Mitigation
Bias mitigation in client selection involves designing strategies to prevent the systematic over- or under-selection of certain client groups, which can lead to unfair or inaccurate global models.
Client Diversity
Client diversity is a selection objective that aims to choose a set of participants whose data distributions and characteristics are representative of the overall population to improve model robustness and generalization.
Federated Coreset
A federated coreset is a small, weighted subset of client data (or synthetic proxy data) that approximates the overall data distribution, used to guide client selection or as a proxy for efficient server-side model updates.
Client Eligibility
Client eligibility refers to the set of pre-defined criteria, such as being online, having sufficient data, or meeting security requirements, that a device must satisfy to be considered for selection in a federated learning round.
Selection Policy
A selection policy is the rule-based, heuristic, or machine learning-based algorithm that governs how clients are chosen for participation in each round of federated learning.
Client Scheduling
Client scheduling is the process of determining not only which clients are selected but also the order and timing of their participation, often using priority queues or deadline-aware algorithms.
Asynchronous FL Selection
Asynchronous FL selection is a client selection paradigm for asynchronous federated learning, where clients are selected and their updates are aggregated as soon as they are available, without waiting for a synchronized round.
Client Dropout
Client dropout refers to the phenomenon where a selected client fails to complete its local training and return an update, a critical factor that selection and aggregation algorithms must be designed to handle robustly.
Privacy-Preserving Selection
Privacy-preserving selection encompasses client selection methods that protect sensitive metadata about clients (e.g., data size, availability) during the selection process using techniques like secure multi-party computation or differential privacy.
Byzantine Client
A Byzantine client is a malicious or faulty participant in a federated learning system that may send incorrect or adversarial model updates, requiring robust selection and aggregation mechanisms for detection and mitigation.
Federated Optimization Techniques
Terms related to specialized optimization methods designed for the federated learning setting. Target: ML Engineers & Researchers.
Federated Averaging (FedAvg)
Federated Averaging (FedAvg) is the foundational iterative optimization algorithm for federated learning, where a central server aggregates locally computed model updates from a subset of clients by taking a weighted average to produce a new global model.
FedProx
FedProx is a federated optimization algorithm that modifies the local client objective by adding a proximal term to constrain local updates, thereby improving convergence and stability when dealing with statistical and systems heterogeneity across clients.
SCAFFOLD
SCAFFOLD (Stochastic Controlled Averaging for Federated Learning) is an optimization algorithm that uses control variates to correct for client drift caused by data heterogeneity, leading to significantly faster convergence compared to standard Federated Averaging.
FedOpt
FedOpt is a framework for federated optimization that generalizes the server-side update step of Federated Averaging, allowing the use of adaptive optimizers like Adam, Yogi, or Adagrad on the global model instead of simple averaging.
FedAdam
FedAdam is a federated optimization algorithm within the FedOpt framework that applies the Adam adaptive optimizer to the server's aggregation of client updates, improving performance on non-convex problems.
FedYogi
FedYogi is a federated adaptive optimization algorithm that adapts the Yogi optimizer for server-side aggregation, offering more stable convergence than FedAdam, particularly in the presence of noisy client gradients.
FedAdagrad
FedAdagrad is a federated optimization algorithm that applies the Adagrad adaptive learning rate method during the server's model aggregation step, assigning smaller updates to frequently occurring features in the global model.
Local Stochastic Gradient Descent (Local SGD)
Local Stochastic Gradient Descent (Local SGD) is the core client-side training procedure in federated learning, where each selected device performs multiple iterations of SGD on its local dataset before communicating its model update to the server.
Client Drift
Client drift is a phenomenon in federated learning where local client models diverge from the global objective due to performing multiple steps of optimization on statistically heterogeneous (non-IID) local data, hindering global convergence.
Adaptive Federated Optimization
Adaptive Federated Optimization refers to a class of federated learning algorithms that incorporate adaptive learning rate methods, such as Adam or Adagrad, either on the server, client, or both sides to improve convergence speed and stability.
Gradient Compression
Gradient compression is a communication-efficient technique in federated learning that reduces the size of model updates sent from clients to the server using methods like sparsification, quantization, or low-rank approximations.
Quantized Gradient Communication
Quantized Gradient Communication is a compression technique for federated learning where high-precision gradient values are mapped to a lower-bit representation before transmission to drastically reduce communication bandwidth.
Top-k Sparsification
Top-k Sparsification is a gradient compression method where only the k largest magnitude values (by absolute value) in a gradient tensor are transmitted, while the rest are set to zero, significantly reducing communication cost.
Error Feedback
Error Feedback is a mechanism used in conjunction with gradient compression techniques that accumulates the compression error locally and adds it back to the next gradient computation, preserving convergence guarantees.
Probabilistic Client Participation
Probabilistic Client Participation is a client sampling strategy in federated learning where devices are selected for each training round based on a probability distribution, often weighted by data quantity or system readiness.
Active Client Selection
Active Client Selection is a strategic approach in federated learning where the server chooses participants for a training round based on criteria designed to improve learning efficiency, such as data quality, resource availability, or update significance.
Asynchronous Federated Optimization
Asynchronous Federated Optimization is a training paradigm where the central server updates the global model immediately upon receiving an update from any client, without waiting for a synchronized round, improving efficiency in heterogeneous environments.
FedAsync
FedAsync is an asynchronous federated learning algorithm where the server aggregates a client's stale update using a mixing hyperparameter that decays with the update's age, mitigating the negative effects of system asynchrony.
Personalized Learning Rates
Personalized Learning Rates in federated learning involve assigning different learning rate schedules or values to individual clients based on their local data distribution or computational characteristics to improve personalized model performance.
Heterogeneous Client Optimization
Heterogeneous Client Optimization refers to federated learning algorithms and strategies specifically designed to handle variations in client data distributions (statistical heterogeneity), hardware capabilities, and network connectivity.
Federated Variance Reduction
Federated Variance Reduction encompasses techniques adapted from classical optimization, like SVRG or SAGA, to reduce the variance of stochastic gradients in the federated setting, accelerating convergence under data heterogeneity.
Federated SVRG
Federated SVRG is an adaptation of the Stochastic Variance Reduced Gradient (SVRG) algorithm for federated learning, which uses control variates to reduce gradient variance and improve convergence speed in heterogeneous environments.
Federated Mirror Descent
Federated Mirror Descent is a generalization of federated optimization that performs gradient updates in a dual space defined by a Bregman divergence, offering a unified framework for handling constraints and non-Euclidean geometry.
Federated Second-Order Optimization
Federated Second-Order Optimization refers to methods that incorporate curvature information (via the Hessian or its approximations) into the federated learning update to achieve faster convergence, though at increased computational and communication cost.
Federated Natural Gradient
Federated Natural Gradient is an optimization method that preconditions client gradients using (an approximation of) the Fisher information matrix, providing an update direction that accounts for the geometry of the model's probability distribution.
Federated Meta-Learning
Federated Meta-Learning applies meta-learning principles, such as Model-Agnostic Meta-Learning (MAML), within a federated framework to learn a global model initialization that can be rapidly adapted to new clients with minimal local data.
Federated Hyperparameter Optimization
Federated Hyperparameter Optimization is the process of tuning model and algorithm hyperparameters (e.g., learning rates, local epochs) in a federated learning system without centralizing client data, often using Bayesian optimization or population-based methods.
Federated Multi-Task Learning
Federated Multi-Task Learning is a paradigm where a federated system learns a set of related but distinct tasks simultaneously across clients, sharing representations to improve individual task performance while respecting data locality.
Federated Reinforcement Learning Optimization
Federated Reinforcement Learning Optimization involves training reinforcement learning policies in a federated manner, where multiple agents interact with different environments and share policy updates to learn a robust global policy.
Communication-Efficient Federated Learning
Terms related to techniques for reducing the bandwidth and frequency of communication between clients and the server. Target: System Architects & CTOs.
Gradient Sparsification
Gradient sparsification is a communication compression technique in federated learning where only a small, critical subset of gradient values (e.g., the largest by magnitude) are transmitted from clients to the server, drastically reducing the size of each update.
Gradient Quantization
Gradient quantization is a communication compression technique in federated learning where high-precision gradient values (e.g., 32-bit floats) are mapped to a lower-bit representation (e.g., 8-bit integers) before transmission, reducing the number of bits required per update.
Error Feedback
Error feedback is a mechanism in communication-efficient federated learning where the compression error (the difference between the original and compressed gradient) is stored locally by the client and added to the next round's gradient before compression, preserving convergence guarantees.
SignSGD
SignSGD is an extreme form of gradient quantization where each client transmits only the sign (+1 or -1) of each gradient component to the server, which then aggregates by majority vote, reducing communication cost to a single bit per parameter.
Sparse Ternary Compression (STC)
Sparse Ternary Compression (STC) is a communication compression technique that combines sparsification and ternary quantization, transmitting only a sparse set of gradient values, each quantized to one of three levels {−α, 0, +α}.
Federated Dropout
Federated dropout is a structured update technique where, in each communication round, a random subset of the model's neurons or layers is 'dropped' (not updated), forcing clients to train and transmit updates for only a fraction of the full model, reducing communication payload size.
Low-Rank Approximation
Low-rank approximation is a model update compression technique where a client's high-dimensional gradient or model update is approximated by the product of two low-rank matrices, significantly reducing the number of values that must be communicated.
Random Projection
Random projection is a sketching technique for dimensionality reduction where a high-dimensional model update is projected onto a random lower-dimensional subspace before transmission, leveraging the Johnson-Lindenstrauss lemma to preserve geometric relationships.
Knowledge Distillation in Federated Learning
Knowledge distillation in federated learning is a communication-efficient technique where clients train local models and send compact 'knowledge' (e.g., softened output logits or embeddings) to the server, which distills this into a global model, avoiding the transmission of full parameter updates.
One-Shot Federated Learning
One-shot federated learning is an extreme communication-efficient paradigm where each client performs local training only once and sends its final model to the server for a single aggregation round, eliminating iterative communication at the cost of model performance.
Asynchronous Federated Learning
Asynchronous federated learning is a communication protocol where the central server aggregates client updates as soon as they arrive, without waiting for a synchronized round, improving efficiency at the cost of potential staleness in aggregated updates.
Staleness-Aware Aggregation
Staleness-aware aggregation is a technique in asynchronous federated learning where the server weights client updates based on their 'staleness' (how many global rounds have passed since the client's model was downloaded), often discounting older updates to maintain convergence stability.
FedProx
FedProx is a federated optimization algorithm that adds a proximal term to the local client objective function, penalizing large deviations from the global model, which helps stabilize training with heterogeneous data and compressed or infrequent communication.
Decentralized Federated Learning
Decentralized federated learning is a peer-to-peer communication architecture where clients (or edge servers) collaborate by directly exchanging model updates with their neighbors via gossip protocols, eliminating the need for a central aggregation server.
Gossip Protocol
A gossip protocol is a decentralized communication scheme used in federated learning where nodes (clients) periodically exchange and average model parameters with a randomly selected subset of peers, eventually achieving global consensus without a central coordinator.
Hierarchical Federated Learning
Hierarchical federated learning is a communication-efficient architecture that introduces intermediate edge servers or cluster heads that perform local aggregation on updates from a subset of clients before forwarding a consolidated update to the central cloud server.
Over-the-Air Computation (AirComp)
Over-the-Air Computation (AirComp) is a wireless communication technique for federated learning where multiple clients simultaneously transmit their analog-modulated model updates over the same radio channel, allowing the receiver to directly compute the sum (aggregation) in the air due to waveform superposition.
Gradient Coding
Gradient coding is a straggler mitigation technique that introduces redundant computation across clients, enabling the server to recover the full gradient aggregate from a subset of completed client updates, thus tolerating slow or failed devices without re-computation.
Partial Participation
Partial participation is a fundamental characteristic of federated learning where only a subset of available clients is selected to participate in each training round, a necessity driven by system constraints that inherently limits communication volume.
Adaptive Client Selection
Adaptive client selection is a communication-aware strategy that dynamically chooses which clients participate in a federated round based on factors like available bandwidth, computational resources, or data utility, to maximize learning efficiency per communication cost.
SCAFFOLD
SCAFFOLD (Stochastic Controlled Averaging for Federated Learning) is an algorithm that uses control variates (correction terms stored on the server and clients) to reduce the variance in client updates caused by data heterogeneity, enabling faster convergence with fewer communication rounds.
Client Drift
Client drift is a phenomenon in federated learning where local client models diverge from the global objective due to multiple steps of local Stochastic Gradient Descent on non-IID data, which is a primary challenge that communication-efficient techniques must mitigate.
Communication Rounds
Communication rounds are the fundamental iterative unit in federated learning, each consisting of a server-to-client model broadcast, local client training, and client-to-server update aggregation; the total number of rounds is a key metric for communication efficiency.
Uplink Communication
Uplink communication in federated learning refers to the transmission of model updates from distributed clients back to the central server, which is typically the bottleneck due to asymmetric network bandwidth and is the primary target of compression techniques.
Model Broadcast
Model broadcast is the downlink phase in a federated learning communication round where the central server transmits the current global model parameters to a selected cohort of clients, constituting a significant portion of the total communication cost.
Communication Complexity
Communication complexity in federated learning is a theoretical measure of the total number of bits or messages that must be exchanged between clients and server to achieve a model of a certain accuracy, which is the core metric optimized by communication-efficient algorithms.
Quantization Noise
Quantization noise is the distortion introduced into a model update when gradient quantization is applied, representing the error between the original full-precision value and its quantized counterpart; effective compression schemes must manage this noise to ensure convergence.
Gradient Clipping
Gradient clipping is a technique that bounds the norm of gradients before compression or transmission, which stabilizes training, controls the dynamic range for quantization, and is a prerequisite for many differential privacy mechanisms in federated learning.
Secure Aggregation Protocols
Terms related to cryptographic methods for combining client model updates without revealing individual contributions. Target: Security Engineers & CTOs.
Secure Aggregation
Secure Aggregation is a cryptographic protocol in federated learning that allows a central server to compute the sum of client model updates without learning any individual client's contribution.
Secure Multi-Party Computation (MPC)
Secure Multi-Party Computation is a cryptographic subfield that enables multiple parties to jointly compute a function over their private inputs while revealing only the final output.
Homomorphic Encryption
Homomorphic Encryption is a form of encryption that allows computations to be performed directly on ciphertext, generating an encrypted result that, when decrypted, matches the result of operations performed on the plaintext.
Differential Privacy
Differential Privacy is a rigorous mathematical framework for quantifying and limiting the privacy loss incurred by an individual when their data is included in a statistical analysis or machine learning model.
Local Differential Privacy (LDP)
Local Differential Privacy is a model of privacy where each client perturbs their data locally before sending it to a central aggregator, providing a strong, trust-minimized privacy guarantee.
Additive Secret Sharing
Additive Secret Sharing is a cryptographic technique where a secret value is split into multiple shares that sum to the original secret, and no subset of shares reveals any information about the secret unless all are combined.
Paillier Cryptosystem
The Paillier Cryptosystem is an additive homomorphic public-key encryption scheme that allows the sum of encrypted values to be computed without decryption.
Fully Homomorphic Encryption (FHE)
Fully Homomorphic Encryption is a type of homomorphic encryption that supports an unlimited number of arbitrary computations (both addition and multiplication) on encrypted data.
Learning With Errors (LWE)
Learning With Errors is a foundational computational problem in lattice-based cryptography that forms the security basis for many post-quantum and homomorphic encryption schemes.
Zero-Knowledge Proof (ZKP)
A Zero-Knowledge Proof is a cryptographic protocol where one party (the prover) can prove to another party (the verifier) that a statement is true without revealing any information beyond the validity of the statement itself.
zk-SNARK
zk-SNARK (Zero-Knowledge Succinct Non-Interactive Argument of Knowledge) is a form of zero-knowledge proof that is extremely short and fast to verify, requiring no interaction between prover and verifier after an initial setup.
Byzantine Robust Aggregation
Byzantine Robust Aggregation refers to algorithms for securely aggregating model updates in federated learning that are resilient to a bounded number of malicious clients sending arbitrary or adversarial updates.
Trusted Execution Environment (TEE)
A Trusted Execution Environment is a secure, isolated area within a main processor that guarantees the confidentiality and integrity of code and data loaded inside it, even from a compromised operating system.
Intel SGX
Intel Software Guard Extensions is a set of security-related instruction codes that create trusted execution environments (enclaves) within Intel CPUs for isolating sensitive application code and data.
Federated Averaging (FedAvg)
Federated Averaging is the foundational iterative algorithm for federated learning where a central server averages model updates from a subset of clients to produce a new global model.
Bonawitz Protocol
The Bonawitz Protocol, often called the Practical Secure Aggregation protocol, is a specific secure aggregation scheme designed for federated learning that uses pairwise masking and is tolerant to client dropouts.
Pairwise Masking
Pairwise Masking is a cryptographic technique used in secure aggregation where each client adds a secret mask shared with another client, such that all masks cancel out when contributions are summed.
Key Agreement Protocol
A Key Agreement Protocol is a cryptographic method that allows two or more parties to establish a shared secret key over an insecure communication channel.
Gaussian Mechanism
The Gaussian Mechanism is a method for achieving differential privacy by adding calibrated Gaussian noise to the output of a function, typically used when the output is a real-valued vector.
Privacy Budget (Epsilon ε)
In differential privacy, the privacy budget (epsilon) is a non-negative parameter that quantifies the maximum allowable privacy loss, with smaller values indicating stronger privacy guarantees.
Verifiable Secret Sharing (VSS)
Verifiable Secret Sharing is an extension of secret sharing that allows shareholders to verify the consistency of their shares, preventing a malicious dealer from distributing inconsistent shares.
Shamir's Secret Sharing
Shamir's Secret Sharing is a threshold secret sharing scheme where a secret is divided into parts, and a minimum number of parts (the threshold) is required to reconstruct the original secret.
Merkle Tree
A Merkle Tree is a cryptographic data structure used to efficiently and securely verify the contents of large datasets, where each leaf node is a hash of a data block and each non-leaf node is a hash of its child nodes.
Message Authentication Code (MAC)
A Message Authentication Code is a short piece of information used to authenticate a message and to provide integrity and authenticity assurances, confirming the message came from the stated sender and was not altered.
Symmetric Encryption
Symmetric Encryption is a type of encryption where the same cryptographic key is used for both the encryption of plaintext and the decryption of ciphertext.
Honest-but-Curious Adversary
An Honest-but-Curious (or semi-honest) adversary is a security model where participants follow the protocol correctly but may attempt to learn additional information from the messages they receive.
Malicious Adversary Model
The Malicious Adversary Model is a strong security model in cryptography where adversaries can arbitrarily deviate from the protocol specification to compromise security.
Gradient Clipping
Gradient Clipping is a technique used in training machine learning models, particularly with differential privacy, to bound the norm of individual gradients, controlling each update's sensitivity and limiting its influence.
Secure Channel
A Secure Channel is a means of data transmission that provides confidentiality, integrity, and often authentication through the use of cryptographic protocols like Transport Layer Security (TLS).
Post-Quantum Cryptography
Post-Quantum Cryptography refers to cryptographic algorithms designed to be secure against attacks by both classical and quantum computers, particularly those leveraging Shor's algorithm.
Differential Privacy in Federated Learning
Terms related to the application of differential privacy mechanisms to client updates for formal privacy guarantees. Target: Privacy Engineers & Compliance Officers.
Differential Privacy (DP)
Differential privacy is a formal mathematical framework for quantifying and limiting the privacy loss incurred by an individual when their data is included in a statistical analysis or machine learning algorithm.
Epsilon (ε)
Epsilon (ε) is the primary privacy parameter in differential privacy that quantifies the maximum allowable privacy loss, where a smaller ε indicates a stronger privacy guarantee.
Delta (δ)
Delta (δ) is a secondary privacy parameter in (ε, δ)-differential privacy that represents a small probability of the privacy guarantee failing, typically set to a value smaller than the inverse of the dataset size.
(ε, δ)-Differential Privacy
(ε, δ)-Differential privacy is a relaxation of pure differential privacy (ε-DP) that allows a small, quantifiable probability δ of the privacy guarantee being violated, enabling more practical noise addition mechanisms.
Local Differential Privacy (LDP)
Local differential privacy is a model where each data owner perturbs their own data with a differentially private mechanism before sending it to an untrusted data curator, providing a strong client-side privacy guarantee.
Central Differential Privacy
Central differential privacy is a model where a trusted curator holds the raw dataset and applies a differentially private mechanism before releasing any query results or aggregated statistics.
Gaussian Mechanism
The Gaussian mechanism is a fundamental algorithm for achieving (ε, δ)-differential privacy by adding calibrated Gaussian noise to the output of a function, where the noise scale is proportional to the function's L2 sensitivity.
Laplace Mechanism
The Laplace mechanism is a fundamental algorithm for achieving pure ε-differential privacy by adding calibrated Laplace noise to the output of a function, where the noise scale is proportional to the function's L1 sensitivity.
Privacy Budget
A privacy budget is a finite allocation of the privacy parameter ε (and sometimes δ) that is consumed over the course of multiple queries or training steps, enforcing a limit on total privacy loss.
Privacy Accounting
Privacy accounting is the process of tracking the cumulative privacy loss (ε, δ) across multiple, potentially adaptive, applications of differentially private mechanisms to ensure the total does not exceed a pre-defined privacy budget.
Composition Theorems
Composition theorems are mathematical rules that quantify how privacy parameters (ε, δ) degrade or compose when multiple differentially private mechanisms are applied sequentially to the same dataset.
Rényi Differential Privacy (RDP)
Rényi differential privacy is a relaxation of differential privacy defined using Rényi divergence, which often provides tighter privacy loss accounting under composition, especially for iterative algorithms like DP-SGD.
Zero-Concentrated Differential Privacy (zCDP)
Zero-concentrated differential privacy is a variant of differential privacy that provides a cleaner analysis of composition for Gaussian-based mechanisms by focusing on the privacy loss random variable's subgaussian tail.
Privacy Amplification
Privacy amplification is a phenomenon where applying a differentially private mechanism to a random subset of data (subsampling) or shuffling client reports can result in a stronger privacy guarantee (smaller effective ε) than analyzing the full dataset.
Amplification by Sampling
Amplification by sampling is a technique where the privacy guarantee of a mechanism is strengthened (amplified) because it is applied only to a randomly chosen subset of the data, such as a mini-batch in stochastic gradient descent.
Client-Level Differential Privacy
Client-level differential privacy is the granularity of protection in federated learning where the privacy guarantee ensures the participation or data of any single client (device/user) cannot be inferred from the released model or aggregate statistics.
Sensitivity (L1, L2)
Sensitivity is the maximum possible change in the output of a function (e.g., a query or gradient) when a single individual's data is added or removed from the dataset, which directly determines the scale of noise required for differential privacy.
DP-SGD (Differentially Private Stochastic Gradient Descent)
DP-SGD is a training algorithm that modifies standard stochastic gradient descent to provide differential privacy by clipping per-example gradients to bound their sensitivity and adding calibrated Gaussian noise to the averaged batch gradient.
DP-FedAvg (Differentially Private Federated Averaging)
DP-FedAvg is a federated learning algorithm that provides client-level differential privacy by having clients clip their local model update (e.g., weight delta or gradient), add calibrated noise, and then send the noised update to the server for secure aggregation.
Gradient Clipping
Gradient clipping is a preprocessing step in differentially private optimization where the norm (L2 or L1) of individual gradients is bounded by a threshold, which limits their sensitivity and controls the scale of noise that must be added.
Clip Threshold
The clip threshold (often denoted C) is the maximum allowed norm for an individual gradient or model update in differentially private training, which is a hyperparameter that trades off bias (from clipping) and variance (from noise).
Noisy Aggregation
Noisy aggregation is a core technique in private federated learning where calibrated random noise is added to the sum or average of client model updates before the aggregated result is revealed, ensuring differential privacy.
Post-Processing Immunity
Post-processing immunity is a fundamental property of differential privacy stating that any function applied to the output of a differentially private mechanism cannot weaken its privacy guarantee, allowing for safe further analysis of private outputs.
Privacy-Preserving Federated Learning
Privacy-preserving federated learning is an umbrella term for techniques, including differential privacy and secure multiparty computation, applied within the federated learning paradigm to protect sensitive client data during collaborative model training.
Moment Accountant
The moment accountant is an advanced privacy accounting method used in DP-SGD that tracks a bound on the moments of the privacy loss random variable to compute tight overall (ε, δ) guarantees under composition.
Exponential Mechanism
The exponential mechanism is a fundamental differentially private algorithm for selecting a high-utility output from a discrete set (like the best candidate in a auction or the top-k items), where the probability of choosing an output is proportional to its exponentiated utility score.
Randomized Response
Randomized response is a classic survey technique and a simple mechanism for achieving local differential privacy, where a respondent answers a sensitive question truthfully with a known probability or lies with the remaining probability.
Federated Learning with Non-IID Data
Terms related to the challenges and solutions for training models on statistically heterogeneous data across clients. Target: ML Engineers & Researchers.
Non-IID Data
Non-Independent and Identically Distributed (Non-IID) data refers to a statistical property where data samples across different clients or sources are not drawn from the same underlying probability distribution, violating the standard assumption of many centralized machine learning algorithms.
Statistical Heterogeneity
Statistical heterogeneity is the fundamental challenge in federated learning where the data distributions across participating clients vary significantly, encompassing differences in feature distributions, label distributions, and data quantities.
Client Drift
Client drift is the phenomenon in federated learning where local client models, trained on their unique Non-IID data, diverge from the global objective, causing convergence instability and degradation of the aggregated global model's performance.
FedProx
FedProx is a federated optimization algorithm that modifies the local client objective by adding a proximal term to penalize updates that stray too far from the current global model, thereby mitigating the negative effects of statistical and system heterogeneity.
SCAFFOLD
SCAFFOLD (Stochastic Controlled Averaging for Federated Learning) is an algorithm that uses control variates to correct for client drift, reducing the variance between local updates and enabling faster, more stable convergence under Non-IID data conditions.
FedOpt
FedOpt (Adaptive Federated Optimization) is a framework that generalizes federated averaging by allowing the server to apply adaptive optimizers like Adam, Yogi, or Adagrad to the aggregated client updates, improving convergence in heterogeneous settings.
Personalized Federated Learning
Personalized Federated Learning (PFL) is a family of techniques designed to produce models tailored to individual clients' local data distributions, rather than a single global model, to address performance disparities caused by Non-IID data.
Ditto
Ditto is a personalized federated learning method that trains a global model alongside personalized local models by incorporating a regularization term that encourages personalization while benefiting from collaborative training.
Clustered Federated Learning
Clustered Federated Learning is an approach where clients are grouped into clusters based on data similarity, and a separate global model is trained for each cluster, effectively handling Non-IID data by creating multiple, more homogeneous model groups.
Federated Distillation (FD)
Federated Distillation is a decentralized training paradigm where clients train local models and share knowledge (e.g., logits or soft labels) instead of model parameters, which can be more communication-efficient and robust to Non-IID data.
Federated Batch Normalization (FedBN)
Federated Batch Normalization is a technique where clients keep their local batch normalization layers private and do not share their statistics during aggregation, which helps to mitigate feature shift caused by Non-IID data across clients.
Federated Learning with Graph Data
Federated Learning with Graph Data extends the federated paradigm to graph-structured data, where Non-IID challenges arise from heterogeneous subgraph topologies, node features, and relational structures across clients.
Dirichlet Distribution Sampling
Dirichlet Distribution Sampling is a standard method for synthetically generating Non-IID data splits in federated learning benchmarks by partitioning a dataset across clients according to a Dirichlet distribution, controlled by a concentration parameter α.
Gradient Diversity
Gradient diversity is a metric that quantifies the directional variation of gradients computed on different clients' data, where low diversity (high alignment) can accelerate convergence but may also indicate homogeneity, while high diversity is characteristic of Non-IID settings.
Bounded Gradient Dissimilarity
Bounded Gradient Dissimilarity is a common theoretical assumption in federated learning convergence analyses that limits the maximum difference between local client gradients and the global gradient, providing a quantifiable measure of data heterogeneity.
Federated Evaluation
Federated Evaluation is the process of assessing the performance of a federated learning model across diverse client-held test sets, which is critical for understanding generalization gaps and fairness under Non-IID data conditions.
Federated Hyperparameter Tuning
Federated Hyperparameter Tuning is the decentralized search for optimal model and training hyperparameters across clients, a complex task due to statistical heterogeneity and the inability to access a centralized validation set.
Federated Learning Benchmarks (LEAF)
Federated Learning Benchmarks, such as the LEAF framework, provide standardized datasets, realistic Non-IID data partitions, and evaluation tools to fairly compare algorithms designed for statistical heterogeneity.
Federated Learning Simulators (Flower, FedML)
Federated Learning Simulators like Flower and FedML are software frameworks that enable researchers and engineers to prototype, simulate, and evaluate federated algorithms, including complex Non-IID data scenarios, in controlled environments.
Federated Learning with Imbalanced Data
Federated Learning with Imbalanced Data addresses the compounded challenge of class imbalance within individual clients' local datasets, which is a specific and common form of Non-IID data skew.
Federated Learning with Sparse Updates
Federated Learning with Sparse Updates employs techniques like Top-k gradient sparsification or Sparse Ternary Compression (STC) to reduce communication costs, which must remain effective despite the gradient bias introduced by Non-IID data.
Byzantine-Robust Federated Learning
Byzantine-Robust Federated Learning encompasses aggregation rules like Krum and Median, designed to tolerate malicious clients, a challenge exacerbated when malicious updates are difficult to distinguish from benign but divergent updates caused by Non-IID data.
Federated Learning with Streaming Data
Federated Learning with Streaming Data, or Online Federated Learning, deals with clients receiving continuous, non-stationary data streams, introducing temporal heterogeneity and concept drift on top of cross-client statistical heterogeneity.
Federated Learning with Model Heterogeneity
Federated Learning with Model Heterogeneity allows clients to train different neural network architectures locally, which can be necessary due to system constraints but complicates aggregation, especially under Non-IID data.
Federated Learning with Fairness Constraints
Federated Learning with Fairness Constraints involves optimizing not just for average global accuracy but also for equitable performance across all clients, a critical objective when performance disparities are widened by Non-IID data.
Personalized Federated Learning
Terms related to techniques for producing models tailored to individual clients' data distributions. Target: ML Engineers & Product Managers.
Personalized Federated Learning (PFL)
Personalized Federated Learning (PFL) is a decentralized machine learning paradigm where a global model is collaboratively trained across multiple clients, but the final deployed model is tailored or adapted to the unique data distribution of each individual client.
Personalized Federated Averaging (pFedAvg)
Personalized Federated Averaging (pFedAvg) is a foundational PFL algorithm that modifies the standard Federated Averaging (FedAvg) process by incorporating client-specific parameters or performing local fine-tuning to produce personalized models for each participant.
Local Fine-Tuning
Local fine-tuning is a PFL technique where a global model, received from a federated learning server, is further trained on a client's local dataset to adapt it to the client's specific data distribution before deployment.
Client-Specific Models
Client-specific models are the end-product of personalized federated learning, referring to the unique model instances, derived from a shared global foundation, that are optimized for the statistical properties of an individual client's data.
Personalization Layers
Personalization layers are specific components of a neural network model (e.g., the final classification head) that are kept local to each client and trained only on local data, while the rest of the model's layers are shared and learned collaboratively.
Meta-Learning for PFL
Meta-learning for PFL applies meta-learning algorithms, such as Model-Agnostic Meta-Learning (MAML), to learn a global model initialization that can be rapidly adapted to new clients with only a few steps of local training.
Model Interpolation
Model interpolation is a PFL strategy where a client's final personalized model is created by computing a weighted average between the global federated model and a model trained solely on the client's local data.
Multi-Task Federated Learning
Multi-Task Federated Learning frames the PFL problem as a multi-task learning problem, where each client's learning objective is treated as a related but distinct task, and the goal is to learn a set of models that share knowledge while performing well on their individual tasks.
Per-Client Hyperparameters
Per-client hyperparameters refer to the practice of allowing or learning unique training hyperparameters (e.g., learning rates, regularization strengths) for each client in a federated learning system to account for local data heterogeneity.
Personalized Federated Optimization
Personalized federated optimization encompasses a class of optimization algorithms designed specifically for the PFL setting, which aim to efficiently find a set of personalized model parameters that perform well across all heterogeneous clients.
Mixture of Experts (MoE) for PFL
A Mixture of Experts (MoE) for PFL is an architecture where a global set of expert sub-models is learned collaboratively, and a client-specific gating network selects or combines these experts to form a personalized model for local inference.
Clustered Federated Learning
Clustered federated learning is a PFL-adjacent approach where clients are partitioned into clusters based on the similarity of their data distributions, and a separate global model is learned and personalized for each cluster.
Personalized Model Aggregation
Personalized model aggregation is a server-side PFL technique where the server, instead of performing a simple average, computes a customized aggregate update for each client based on the client's relationship to others or its update history.
FedPer
FedPer is a canonical PFL algorithm that personalizes models by keeping the base layers of a neural network global (shared and federated) while the personalized layers (typically the task-specific head) are local and never aggregated.
FedRep
FedRep is a PFL algorithm that learns a global representation (feature extractor) across all clients while learning unique local heads (classifiers) for each client, decoupling shared feature learning from client-specific decision boundaries.
Hypothesis Transfer Learning
Hypothesis transfer learning in PFL involves using the global model as a source hypothesis, which is then transferred and adapted locally by each client, often with regularization to prevent catastrophic forgetting of useful shared knowledge.
Personalized Federated Meta-Learning (PFML)
Personalized Federated Meta-Learning (PFML) is a framework that combines federated learning with meta-learning to learn a global model that is explicitly optimized for fast adaptation to new clients with limited local data.
Contextual PFL
Contextual PFL incorporates client context (e.g., device type, location, user demographics) into the personalization process, using this auxiliary information to guide or condition the model adaptation for more relevant personalization.
Client Drift Compensation
Client drift compensation refers to techniques in PFL that mitigate the phenomenon where local client training diverges significantly from the global objective due to data heterogeneity, often through regularization or controlled aggregation.
Personalized Gradient Descent
Personalized gradient descent modifies the local optimization step for each client, for example by using a client-specific gradient correction term, to steer local updates towards a beneficial personalized optimum rather than the global one.
Layer-wise Personalization
Layer-wise personalization is a design strategy in PFL where specific layers of a neural network architecture are designated as personalizable (trained locally) or global (trained federated), allowing for architectural control over the personalization granularity.
Personalized Federated Distillation
Personalized federated distillation is a PFL technique where knowledge from a global model or an ensemble of other clients' models is distilled into a client's local model, enabling personalization without directly sharing raw model parameters.
Knowledge Distillation for PFL
Knowledge distillation for PFL employs distillation loss functions during local training to align the predictions or representations of a client's personalized model with those of a more knowledgeable global or teacher model.
Personalized Model Head
A personalized model head is the task-specific final layer(s) of a neural network that are unique to each client in a PFL system, while the earlier feature-extraction layers are shared, as seen in algorithms like FedPer and FedRep.
FedBN (Federated Batch Normalization)
FedBN is a PFL technique that keeps batch normalization layer parameters (mean and variance statistics) local to each client during federated training, thereby accounting for feature distribution shift across clients without aggregating these statistics.
Adaptive Federated Optimization (FedOpt)
Adaptive Federated Optimization (FedOpt) refers to server-side optimization methods, like using Adam or AdaGrad on the server to aggregate client updates, which can be leveraged in PFL to dynamically weight client contributions for better personalized outcomes.
Personalized Federated Learning with Regularization
This approach adds a regularization term to the local client's loss function, typically penalizing the distance between the local personalized model and the global model, to balance personalization with the retention of beneficial shared knowledge.
Client-Centric Aggregation
Client-centric aggregation is a family of server aggregation rules in PFL that weight or transform incoming client updates based on client-specific factors, such as data quality or similarity to other clients, to produce more relevant aggregated models for each participant.
Federated Learning Orchestrators
Terms related to the software platforms and frameworks that manage the federated learning lifecycle. Target: DevOps Engineers & CTOs.
Federated Learning Orchestrator
A Federated Learning Orchestrator is the central software component that manages the lifecycle, coordination, and aggregation of a federated learning job across a distributed network of edge devices or siloed servers.
Central Aggregator
A Central Aggregator is the server-side component in a federated learning system responsible for securely receiving, combining, and averaging model updates from participating clients to produce an improved global model.
Client Manager
A Client Manager is a module within a federated learning orchestrator that handles the registration, authentication, profiling, and lifecycle state of all participating edge devices or clients in the federation.
Model Registry
A Model Registry is a centralized repository within a federated learning system that stores, versions, and manages the metadata and artifacts for global models, client models, and model checkpoints.
Round Coordinator
A Round Coordinator is the component of a federated learning orchestrator that manages the execution of a single training round, including client selection, task dispatch, update collection, and aggregation triggering.
Task Scheduler
A Task Scheduler is a component that determines the order, timing, and resource allocation for federated learning tasks across heterogeneous clients, often implementing policies for fairness, efficiency, and priority.
Client Selection Module
A Client Selection Module is the algorithmic component of a federated learning orchestrator that chooses a subset of available devices to participate in a given training round based on criteria like resource availability, data distribution, or network conditions.
Secure Aggregation Orchestrator
A Secure Aggregation Orchestrator is a specialized component that coordinates cryptographic secure aggregation protocols, ensuring client model updates can be combined without the server learning any individual client's contribution.
Differential Privacy Orchestrator
A Differential Privacy Orchestrator is a system component that manages the application of differential privacy mechanisms, such as noise injection and clipping, to client updates before aggregation to provide formal privacy guarantees.
Federated Job
A Federated Job is a defined machine learning training task in a federated learning system, encompassing the model architecture, training configuration, client selection criteria, and aggregation strategy to be executed over multiple rounds.
Configuration Manager
A Configuration Manager is a service within a federated learning orchestrator that centrally stores, distributes, and validates the hyperparameters, model architectures, and environment settings for all clients and server components.
Convergence Monitor
A Convergence Monitor is a component that tracks the performance and stability of the global model across federated learning rounds, using metrics to determine when training has converged or should be terminated.
Heterogeneity Handler
A Heterogeneity Handler is a system module designed to manage the variability in compute capability, memory, network connectivity, and data distribution across federated learning clients to ensure stable and efficient training.
Fault Tolerance Manager
A Fault Tolerance Manager is a component that implements strategies like checkpointing, retries, and client dropout handling to ensure a federated learning job can complete successfully despite partial client or network failures.
API Gateway (Federated Learning)
An API Gateway in a federated learning system is a single entry point that manages and routes client-server communication, handling authentication, protocol translation, rate limiting, and request/response transformation.
Workflow Engine (Federated Learning)
A Workflow Engine in federated learning automates and sequences the steps of the federated learning lifecycle, such as model initialization, round execution, aggregation, validation, and deployment, often using a directed acyclic graph (DAG).
Edge Inference Manager
An Edge Inference Manager is a component that coordinates the deployment, versioning, and execution of trained federated models on edge devices for local prediction, often integrating with the orchestrator's model registry.
Kubernetes Operator for Federated Learning
A Kubernetes Operator for Federated Learning is a method of packaging, deploying, and managing a federated learning orchestrator and its components on Kubernetes using custom resources and automated control loops.
Cross-Silo Orchestrator
A Cross-Silo Orchestrator is a federated learning coordinator designed for settings where clients are a small number of institutional entities (e.g., hospitals, banks) with relatively powerful, reliable servers and data silos.
Cross-Device Orchestrator
A Cross-Device Orchestrator is a federated learning coordinator designed for settings with a massive number of unreliable, resource-constrained edge devices (e.g., smartphones, IoT sensors), requiring highly scalable and fault-tolerant management.
Hierarchical Aggregation
Hierarchical Aggregation is a federated learning orchestration strategy where client updates are first aggregated at intermediate nodes (e.g., edge servers or regional clusters) before being sent to a central server, reducing communication overhead and latency.
Federated SDK
A Federated SDK is a software development kit provided to client devices, containing libraries and APIs for local model training, secure communication with the orchestrator, and integration with the device's data and compute resources.
Audit Logger
An Audit Logger in a federated learning system is a component that records immutable logs of all significant events, such as client participation, model updates, aggregation actions, and access attempts, for compliance and debugging.
Resource Monitor
A Resource Monitor is a component that collects real-time telemetry on client and server resource utilization (CPU, memory, network, battery) to inform scheduling, load balancing, and fault detection decisions within the federated learning orchestrator.
Compliance Checker
A Compliance Checker is a module that validates federated learning operations against regulatory and policy constraints, such as data residency rules, privacy budgets, or approved model architectures, before allowing execution.
Deployment Manager (Federated Learning)
A Deployment Manager in federated learning handles the promotion, distribution, and rollback of trained global models to production inference endpoints, often coordinating with canary analysis and A/B testing frameworks.
Edge Device Heterogeneity Management
Terms related to handling variations in compute, memory, connectivity, and availability across federated clients. Target: System Architects & Embedded Engineers.
Client Capability Profiling
Client capability profiling is the process of systematically measuring and cataloging the computational resources, memory, network connectivity, and power availability of edge devices participating in a federated learning system.
Resource-Aware Scheduling
Resource-aware scheduling is a federated learning orchestration strategy that dynamically assigns training tasks to edge clients based on their real-time available computational power, memory, and energy constraints.
Dynamic Batching
Dynamic batching is a technique in federated edge learning where the local batch size for on-device training is automatically adjusted per client based on its current memory and compute capacity to prevent out-of-memory errors and optimize throughput.
Adaptive Model Partitioning
Adaptive model partitioning is a method that splits a neural network model into segments, offloading computationally intensive layers to a server or nearby edge node while keeping simpler layers on a resource-constrained device for federated training.
Heterogeneous Federated Averaging (HeteroFA)
Heterogeneous Federated Averaging (HeteroFA) is a variant of the Federated Averaging algorithm designed to aggregate model updates from clients with vastly different computational capabilities, often by weighting contributions or allowing variable local computation.
Compute-Aware Selection
Compute-aware selection is a client selection strategy for federated learning that prioritizes devices with sufficient available processing power to complete a training round within a target latency window, improving system efficiency.
Memory-Constrained Optimization
Memory-constrained optimization refers to techniques, such as gradient checkpointing or selective layer updating, that modify the federated training process to operate within the strict RAM limitations of edge devices like smartphones and IoT sensors.
Stratified Client Sampling
Stratified client sampling is a method for selecting participants in a federated learning round that ensures a representative mix of devices from different capability tiers (e.g., high-end phones, low-end sensors) to mitigate bias from hardware heterogeneity.
Tiered Aggregation
Tiered aggregation is a hierarchical aggregation scheme in federated learning where updates from clients with similar resource profiles are first aggregated locally before being combined into a global model, improving scalability and handling heterogeneity.
Availability-Aware Round Scheduling
Availability-aware round scheduling is the coordination of federated training rounds to align with the predicted or declared active periods of edge devices, accommodating intermittent connectivity and sleep cycles common in mobile and IoT networks.
Connectivity-Aware Compression
Connectivity-aware compression is an adaptive technique that applies more aggressive model update compression (e.g., higher sparsity or quantization) for federated clients with poor or expensive network links to reduce communication overhead.
Elastic Federated Learning
Elastic federated learning is a system design paradigm where the global model architecture, training workload, and participation requirements can dynamically scale up or down to match the collective and varying resources of the available client pool.
Progressive Model Download
Progressive model download is a client-side technique where a global model is fetched and instantiated in stages over multiple communication rounds, allowing resource-constrained devices to begin training with a partial model.
On-Device Resource Monitor
An on-device resource monitor is a lightweight software agent that runs on a federated learning client, continuously tracking metrics like CPU utilization, memory pressure, battery level, and thermal status to inform local training decisions.
Federated Device Registry
A federated device registry is a centralized or distributed database maintained by the federation server that stores the static and dynamic capability profiles, status, and historical performance of all enrolled edge devices.
Capability-Based Pruning
Capability-based pruning is a technique where a global neural network model is pruned to a different sparsity level for each federated client based on its specific compute and memory profile before local training commences.
Variable-Length Training Rounds
Variable-length training rounds is a federated learning protocol that allows clients to perform different numbers of local stochastic gradient descent steps based on their available resources before submitting an update, rather than enforcing a fixed epoch count.
Asynchronous Federated Updates
Asynchronous federated updates is a communication protocol where the server aggregates client model updates as soon as they are received, without waiting for a synchronized round to finish, thereby accommodating clients with highly variable training times.
Partial Model Participation
Partial model participation is a federated training scheme where each client only trains a randomly selected subset of the global model's parameters or layers in each round, significantly reducing per-device computational and memory load.
Layer-Wise Federated Training
Layer-wise federated training is a decentralized training strategy where different layers of a neural network are assigned to and trained by different subsets of clients based on their capabilities, with periodic synchronization.
Federated Dropout
Federated dropout is a technique that randomly drops out a subset of neurons or entire layers for individual clients during a training round, creating smaller sub-models that reduce computation and can improve generalization in heterogeneous systems.
Dynamic Width Networks
Dynamic width networks are neural architectures with adjustable layer sizes that can be scaled down for inference and training on resource-constrained federated clients while maintaining a larger master model on the server for aggregation.
Mixture-of-Experts (MoE) Federated Learning
Mixture-of-Experts (MoE) federated learning is a paradigm where a global model is composed of multiple expert sub-networks, and each federated client only activates and trains a sparse combination of experts suited to its data and capabilities.
Federated Quantization-Aware Training (FQAT)
Federated Quantization-Aware Training (FQAT) is a process that simulates the effects of low-precision arithmetic during the federated training of a model, ensuring the final aggregated model remains accurate when deployed in quantized form on heterogeneous edge hardware.
Per-Client Learning Rate Tuning
Per-client learning rate tuning is an optimization strategy in federated learning where the server or the client itself adjusts the local stochastic gradient descent learning rate based on the device's compute profile, data volume, or update staleness.
Adaptive Federated Optimization (FedOpt)
Adaptive Federated Optimization (FedOpt) is a class of server-side optimization algorithms, like FedAdam or FedYogi, that use adaptive moment estimation to update the global model, often providing more stable convergence across heterogeneous clients than simple averaging.
Federated Hardware Abstraction Layer (HAL)
A Federated Hardware Abstraction Layer (HAL) is a software interface within a federated learning framework that standardizes interactions with diverse edge hardware (CPUs, GPUs, NPUs), allowing training tasks to be deployed without device-specific code.
Battery-Aware Federated Learning
Battery-aware federated learning is a system design principle that modifies client selection, training intensity, and communication frequency to minimize the energy drain on mobile and IoT devices, prioritizing user experience and device longevity.
Thermal-Throttling Management
Thermal-throttling management in federated edge learning involves client-side algorithms that proactively reduce computational load or pause training when device temperature approaches critical levels to prevent hardware damage and performance degradation.
Federated Intermittent Connectivity Protocol
A federated intermittent connectivity protocol is a communication standard that enables edge devices to cache model updates, resume interrupted training sessions, and reliably synchronize with the server over unstable or periodically available network links.
Federated Learning Attack Mitigation
Terms related to defenses against adversarial clients, data poisoning, and model inversion attacks in federated systems. Target: Security Engineers & ML Engineers.
Byzantine Robust Aggregation
Byzantine Robust Aggregation is a class of algorithms designed to produce a correct global model update in federated learning even when a fraction of participating clients are malicious or faulty, sending arbitrary or adversarial updates.
Krum Algorithm
The Krum algorithm is a Byzantine-robust aggregation rule that selects a single client's model update as the global update by choosing the one whose parameter vector is closest, in Euclidean distance, to its nearest neighbors, thereby filtering out outliers.
Bulyan
Bulyan is a meta-aggregation Byzantine defense that first applies a robust aggregation rule like Krum or trimmed mean to select a set of candidate updates, then computes their coordinate-wise trimmed mean to produce the final robust global update.
Trimmed Mean Aggregation
Trimmed Mean Aggregation is a robust statistical method where, for each model parameter dimension, a fraction of the highest and lowest values from client updates are discarded before computing the mean of the remaining values to form the global update.
Median Aggregation
Median Aggregation is a Byzantine-robust federated learning technique where the server computes the coordinate-wise median of all received client model updates to form the new global model, which is highly resilient to extreme outlier values.
Data Poisoning Defense
Data Poisoning Defense in federated learning refers to techniques designed to detect and mitigate attacks where malicious clients manipulate their local training data to corrupt the global model's performance or inject a backdoor.
Backdoor Attack Mitigation
Backdoor Attack Mitigation encompasses strategies to prevent or remove hidden triggers implanted in a federated model by adversarial clients, ensuring the model behaves normally on clean inputs but misclassifies inputs containing the trigger pattern.
Model Inversion Defense
Model Inversion Defense refers to countermeasures against attacks that aim to reconstruct representative samples of a client's private training data by repeatedly querying the shared global model or its updates.
Membership Inference Defense
Membership Inference Defense involves techniques to prevent an adversary from determining whether a specific data record was part of a client's private training dataset by analyzing the global model or its updates.
Sybil Attack Resistance
Sybil Attack Resistance in federated learning refers to mechanisms that prevent a single malicious entity from controlling multiple fake client identities (Sybils) to disproportionately influence the model training process.
Free-Rider Detection
Free-Rider Detection identifies clients in a federated learning system that benefit from the global model without contributing meaningful updates, either due to malice or having non-informative local data.
Trust Scoring
Trust Scoring is a defense mechanism that assigns a dynamic credibility score to each federated client based on the historical quality and consistency of their updates, which is then used to weight their contribution during aggregation.
Gradient Inspection
Gradient Inspection is a server-side defense technique that analyzes the statistics, distribution, or geometry of submitted client model updates (gradients) to detect anomalies indicative of malicious behavior or poor data quality.
Update Sanitization
Update Sanitization is the process of filtering, clipping, or otherwise modifying client-submitted model updates before aggregation to remove potentially malicious components or excessive noise.
Adversarial Training (Federated)
Federated Adversarial Training is a defense technique where clients locally train their models not only on their clean data but also on adversarially perturbed examples, aiming to improve the global model's robustness to evasion attacks.
Local Differential Privacy (LDP)
Local Differential Privacy (LDP) is a privacy model where each client perturbs their data or model update with noise before sending it to the server, providing a strong, distributed privacy guarantee without needing a trusted central aggregator.
Gaussian Mechanism
The Gaussian Mechanism is a differential privacy technique that adds calibrated Gaussian noise to a function's output (like a model update) to provide (epsilon, delta)-differential privacy, commonly used in federated learning for its analytical properties.
Rényi Differential Privacy (RDP)
Rényi Differential Privacy (RDP) is a relaxation of pure differential privacy that uses the Rényi divergence to provide tighter composition bounds, making it a powerful tool for privacy accounting in iterative processes like federated learning.
Secure Enclaves (e.g., SGX, TrustZone)
Secure Enclaves, such as Intel SGX or ARM TrustZone, are hardware-based trusted execution environments (TEEs) that create isolated, encrypted memory regions on a client device to protect code and data (like model updates) during federated learning computations.
Federated Attestation
Federated Attestation is a security protocol where a central server cryptographically verifies that a client's device is running genuine, unmodified software within a trusted execution environment (TEE) before accepting its model updates.
Model Watermarking
Model Watermarking in federated learning embeds a unique, secret signature into the global model during training, allowing the model owner to later verify ownership or identify which client leaked a stolen model copy.
Gradient Noise Addition
Gradient Noise Addition is a privacy-enhancing technique where clients or the server add carefully calibrated random noise to model updates before sharing or aggregation, providing a differential privacy guarantee for the training process.
Privacy Accounting
Privacy Accounting is the systematic tracking of the cumulative privacy loss (epsilon, delta) across multiple training rounds in differentially private federated learning, ensuring the total expenditure stays within a pre-defined privacy budget.
Byzantine Fault Tolerance (BFT)
Byzantine Fault Tolerance (BFT) in federated learning is the property of a distributed system to correctly reach consensus on a global model state despite a subset of participants (clients) behaving arbitrarily or maliciously.
Client-Side Validation
Client-Side Validation refers to defensive checks performed by a federated learning client on its own local data or computed updates before submission, such as checking for data quality or clipping gradient norms to limit influence.
Server-Side Validation
Server-Side Validation encompasses all defensive checks and filtering operations performed by the central aggregator on received client updates before aggregation, including anomaly detection, norm bounding, and statistical tests.
Robust Loss Functions
Robust Loss Functions, such as the Huber loss, are designed to be less sensitive to outliers in the training data, making the local training process on each client more resilient to label noise or data poisoning attacks.
Adversarial Example Detection
Adversarial Example Detection in federated learning involves techniques for identifying maliciously crafted input data designed to fool the trained model, either during local client training or after global model deployment.
Privacy-Preserving Aggregation
Privacy-Preserving Aggregation is a broad category of cryptographic and algorithmic techniques that allow a federated learning server to compute the sum or average of client updates without learning the value of any individual update.
Threat Modeling (Federated)
Federated Learning Threat Modeling is the structured process of identifying potential security and privacy threats (e.g., data poisoning, inference attacks) to a federated learning system, assessing their risk, and defining appropriate mitigation strategies.
Federated Learning for TinyML
Terms related to adapting federated learning paradigms for ultra-constrained microcontrollers and sensors. Target: Embedded Engineers & ML Engineers.
TinyML
Tiny Machine Learning (TinyML) is a subfield of machine learning focused on developing and deploying ultra-low-power, memory-efficient models capable of running inference and, increasingly, training directly on microcontroller units (MCUs) and other deeply embedded, resource-constrained devices.
Microcontroller Unit (MCU)
A Microcontroller Unit (MCU) is a compact, integrated circuit designed to govern a specific operation in an embedded system, combining a processor core, memory, and programmable input/output peripherals on a single chip, forming the primary hardware target for TinyML deployments.
On-Device Training
On-device training is the process of updating a machine learning model's parameters directly on an edge device using locally generated data, as opposed to solely performing inference, enabling continual learning and personalization without raw data leaving the device.
Memory Footprint
Memory footprint refers to the total amount of volatile (RAM) and non-volatile (Flash) memory consumed by a machine learning model's parameters, activations, and runtime buffers, which is a critical constraint for deployment on TinyML devices.
Compute Constraint
Compute constraint refers to the limitation imposed by the available processing power, typically measured in operations per second (OPS) or clock speed, of a resource-constrained device, which restricts the complexity of machine learning models and training algorithms that can be executed.
Energy Budget
Energy budget is the total amount of electrical energy allocated for a computational task, such as model training or inference, which is a fundamental design constraint for battery-powered TinyML devices and directly impacts operational lifetime.
Quantization-Aware Training (QAT)
Quantization-Aware Training (QAT) is a model compression technique where a neural network is trained with simulated low-precision (e.g., 8-bit integer) arithmetic, allowing the model to learn to compensate for the quantization error and maintain higher accuracy compared to post-training quantization.
Post-Training Quantization (PTQ)
Post-Training Quantization (PTQ) is a model compression technique that converts a pre-trained model's weights and activations from high-precision floating-point (e.g., 32-bit) to lower-precision integer (e.g., 8-bit) formats after training is complete, reducing model size and accelerating inference with minimal calibration data.
Weight Pruning
Weight pruning is a model compression technique that removes less important parameters (weights) from a neural network, either by setting them to zero (unstructured pruning) or removing entire structures like neurons or channels (structured pruning), to reduce the model's size and computational cost.
Model Sparsification
Model sparsification is the process of inducing sparsity in a neural network by zeroing out a significant fraction of its parameters (weights or gradients), creating a sparse model that can be stored and computed more efficiently using specialized hardware or software libraries.
Sparse Update
A sparse update in federated learning is a client model update (e.g., gradient or weight delta) where only a small subset of the parameters have non-zero values, a technique used to drastically reduce the communication cost per round in federated edge learning systems.
Low-Precision Arithmetic
Low-precision arithmetic refers to performing mathematical computations, such as those in neural network inference and training, using numerical formats with fewer bits (e.g., 8-bit integers, 16-bit floating-point) than standard 32-bit floating-point, to reduce memory usage, power consumption, and latency.
Integer-Only Inference
Integer-only inference is the execution of a quantized neural network exclusively using integer arithmetic operations, eliminating the need for floating-point hardware and enabling highly efficient deployment on microcontrollers and low-power accelerators.
TinyML Stack
The TinyML stack is the layered software and hardware architecture required to develop, optimize, compile, and deploy machine learning models on ultra-low-power microcontrollers, typically comprising model design frameworks, hardware-aware compilers, microcontroller runtimes, and embedded operating systems.
Embedded FL Runtime
An embedded Federated Learning (FL) runtime is a lightweight software library deployed on a microcontroller or edge device that manages the local execution of the federated learning client protocol, including model updates, secure communication, and resource management.
Firmware Integration
Firmware integration is the process of embedding a machine learning model and its associated inference or training runtime into the low-level system software (firmware) that directly controls an embedded device's hardware, creating a single, deployable binary image.
Over-the-Air Update (OTA)
An Over-the-Air (OTA) update is a method of wirelessly distributing new software, firmware, or machine learning model parameters to deployed edge devices, enabling remote maintenance, bug fixes, and model evolution in federated learning systems.
Resource-Constrained Device
A resource-constrained device is an embedded system, such as a microcontroller-based sensor, with severe limitations in available memory, processing power, energy supply, and network bandwidth, which defines the primary challenge space for TinyML and federated edge learning.
Heterogeneous Clients
Heterogeneous clients in federated learning refer to the variation in hardware capabilities (compute, memory), data distributions, network connectivity, and availability among the participating edge devices, which complicates synchronous training and requires adaptive algorithms.
Availability Window
An availability window is the limited period during which a federated learning client (e.g., a mobile or IoT device) is powered on, connected to a network, and has sufficient idle compute resources to participate in a local training round.
Straggler Problem
The straggler problem in federated learning occurs when a small number of slow or unresponsive clients significantly delay the completion of a synchronous aggregation round, reducing overall system efficiency, a challenge exacerbated by the heterogeneity of edge devices.
Partial Participation
Partial participation is a fundamental characteristic of federated learning at scale, where only a subset of the total client population is available or selected to participate in any given training round, due to constraints like availability windows, connectivity, and client sampling strategies.
TinyML Benchmark Suite
A TinyML benchmark suite is a standardized collection of machine learning models, datasets, and evaluation metrics designed to measure and compare the performance, accuracy, memory usage, and energy consumption of ML frameworks and hardware platforms targeting microcontroller-class devices.
On-Device Dataset
An on-device dataset is the collection of sensor readings, user interactions, or other locally generated data stored on an edge device, which is used for local model training or personalization in federated and on-device learning paradigms without being transmitted to a central server.
Sensor Data Stream
A sensor data stream is a continuous, real-time sequence of measurements (e.g., acceleration, temperature, audio) generated by an embedded sensor, which serves as the primary input for on-device inference and training in TinyML applications.
Cold-Start Problem
The cold-start problem in federated edge learning refers to the challenge of initializing a global model or beginning training when client devices have little to no relevant local data, or when a new device joins the federation, potentially leading to poor initial performance or slow convergence.
Warm-Start Model
A warm-start model is a pre-trained or pre-initialized machine learning model provided to federated learning clients at the beginning of training, used to accelerate convergence, improve stability, and mitigate the cold-start problem compared to random initialization.
On-Device Preprocessing
On-device preprocessing is the execution of data transformation and feature extraction algorithms (e.g., filtering, normalization, Fast Fourier Transform) directly on the edge device, reducing the volume of raw data that must be stored or transmitted and tailoring input for the model.
Thermal Throttling
Thermal throttling is a hardware protection mechanism where a processor reduces its clock speed to lower power consumption and heat generation when a critical temperature threshold is exceeded, which can severely impact the performance and reliability of compute-intensive on-device training.
Battery Drain
Battery drain is the rate at which a device's stored electrical energy is depleted, a critical performance metric for battery-powered TinyML devices where energy-intensive operations like wireless communication and on-device training directly impact operational lifespan.
Federated Model Evaluation Metrics
Terms related to measuring model performance, fairness, and convergence in a decentralized setting. Target: ML Engineers & Researchers.
Global Model Accuracy
Global model accuracy is the performance of the aggregated federated model on a held-out test set that is representative of the overall target population, measured after the model parameters have been averaged across participating clients.
Local Model Accuracy
Local model accuracy is the performance of a model on the private data held by an individual client, which can differ significantly from the global model accuracy due to statistical heterogeneity (non-IID data).
Model Convergence
Model convergence in federated learning refers to the state where the global model's parameters stabilize and its performance on a validation set ceases to improve significantly with further communication rounds.
Client Drift
Client drift is the phenomenon where local models, trained on statistically heterogeneous (non-IID) client data, diverge from the global model objective, leading to slow or unstable convergence of the federated averaging process.
Non-IID Data
Non-IID (Non-Independent and Identically Distributed) data in federated learning describes the statistical heterogeneity where data distributions vary significantly across clients, posing a fundamental challenge to model convergence and performance.
Statistical Heterogeneity
Statistical heterogeneity is the variation in the underlying data distribution (e.g., feature distribution, label distribution) across different clients in a federated learning system, a core challenge addressed by personalized and robust aggregation techniques.
Fairness Metrics
Fairness metrics in federated learning are quantitative measures, such as demographic parity or equalized odds, used to evaluate whether a model's predictions exhibit unwanted bias against specific subgroups defined by sensitive attributes like gender or race.
Differential Privacy (DP)
Differential privacy (DP) is a rigorous mathematical framework that provides formal privacy guarantees by ensuring the inclusion or exclusion of any single individual's data in the training set has a negligible effect on the model's output.
Epsilon (ε)
Epsilon (ε) is the core privacy budget parameter in differential privacy that quantifies the maximum allowable privacy loss, where a smaller ε provides stronger privacy guarantees but typically degrades model utility.
Secure Aggregation
Secure aggregation is a cryptographic protocol that allows a central server in federated learning to compute the sum of client model updates without being able to inspect any individual client's contribution, thus preserving data privacy.
Model Robustness
Model robustness in federated learning refers to the global model's ability to maintain high performance and stability despite the presence of malicious clients (Byzantine robustness) or adversarial input perturbations (adversarial robustness).
Robust Aggregation
Robust aggregation refers to a class of federated averaging algorithms, such as Krum or coordinate-wise median, designed to produce a reliable global model even when a fraction of clients are malicious or submit corrupted updates.
Generalization Gap
The generalization gap in federated learning is the difference in performance between a model on its local training data and its performance on unseen global data, which can be exacerbated by non-IID data distributions across clients.
Personalization Performance
Personalization performance measures how well a federated learning model, after global training, can be adapted or fine-tuned to perform accurately on the specific data distribution of an individual client or device.
Communication Cost
Communication cost in federated learning quantifies the total bandwidth required to transmit model updates between clients and the central server, often a primary bottleneck addressed via compression and sparsification techniques.
Client Contribution
Client contribution evaluation involves quantifying the impact of an individual client's data and updates on the final global model's performance, often measured using techniques like the Shapley value or influence functions.
Federated Evaluation
Federated evaluation is the decentralized process of assessing a model's performance across multiple clients without centralizing their private data, typically involving metrics computed locally and aggregated securely on a server.
Cross-Client Validation
Cross-client validation is a model evaluation technique in federated learning where a model trained on a subset of clients is validated on the held-out data from other clients to estimate its generalization to new, unseen data distributions.
On-Device Evaluation
On-device evaluation is the process of measuring a machine learning model's performance metrics, such as latency, accuracy, and energy consumption, directly on the edge device or microcontroller where it is deployed.
Model Compression
Model compression is a set of techniques, including quantization, pruning, and knowledge distillation, used to reduce the memory footprint and computational requirements of a neural network for efficient deployment on resource-constrained edge devices.
Federated Hyperparameter Tuning
Federated hyperparameter tuning is the process of optimizing model and training parameters, such as learning rate or local epochs, in a decentralized setting without direct access to the union of all clients' private data.
System Heterogeneity
System heterogeneity refers to the variation in computational capability, memory, network connectivity, and availability (stragglers) across client devices in a federated learning system, which impacts training efficiency and requires adaptive orchestration.
Straggler Effect
The straggler effect is the slowdown in federated learning training rounds caused by a subset of clients that are significantly slower to compute and transmit their model updates due to limited hardware, poor connectivity, or background tasks.
Convergence Rate
Convergence rate measures how quickly the global model's loss decreases or its accuracy increases per communication round in federated learning, influenced by factors like client selection, data heterogeneity, and optimization algorithms.
Utility-Privacy Trade-off
The utility-privacy trade-off describes the inverse relationship in federated learning where increasing the strength of privacy guarantees (e.g., via differential privacy) typically reduces the final model's accuracy or utility.
Byzantine Robustness
Byzantine robustness is a property of a federated learning aggregation rule that ensures the global model remains accurate and stable even when a bounded number of clients are malicious and send arbitrarily corrupted model updates.
Membership Inference Attack
A membership inference attack is a privacy attack where an adversary, often with query access to a trained model, attempts to determine whether a specific data record was part of the model's training dataset.
Federated Test Set
A federated test set is a curated, globally representative dataset used to evaluate the final performance of a federated learning model, which may be held centrally by the server or constructed from client-held data via secure protocols.
Model Calibration
Model calibration is the property of a machine learning model where its predicted probabilities of outcomes accurately reflect the true likelihood of those outcomes, often measured using the Expected Calibration Error (ECE).
Federated Transfer Learning
Terms related to leveraging knowledge from a source domain or model to improve learning on federated target tasks. Target: ML Engineers & Researchers.
Federated Transfer Learning
Federated transfer learning is a decentralized machine learning paradigm where knowledge from a source domain or pre-trained model is transferred to improve learning on a target task across distributed clients without sharing raw data.
Cross-Domain Adaptation
Cross-domain adaptation is a transfer learning technique that adjusts a model trained on a source data distribution to perform effectively on a different, but related, target data distribution within a federated learning framework.
Heterogeneous Transfer Learning
Heterogeneous transfer learning addresses scenarios where the source and target tasks or data modalities differ significantly, requiring specialized techniques to align feature spaces or model architectures in a federated setting.
Model Warm-Starting
Model warm-starting is the practice of initializing a federated learning model with parameters from a pre-trained source model to accelerate convergence and improve final performance on the target task.
Continual Federated Learning
Continual federated learning is a paradigm where a model learns sequentially from a stream of non-stationary data across distributed clients while mitigating catastrophic forgetting of previously learned tasks.
Meta-Learning for Federated Learning
Meta-learning for federated learning involves training a model's initialization or learning algorithm on a distribution of related federated tasks so it can adapt quickly to new clients or tasks with minimal data.
Few-Shot Federated Learning
Few-shot federated learning enables a model to learn new concepts or tasks from only a handful of labeled examples per client by leveraging knowledge transferred from a related source domain.
Zero-Shot Transfer Learning
Zero-shot transfer learning enables a federated model to perform a target task for which it has seen no labeled examples during training, typically by leveraging semantic relationships or auxiliary information learned from a source domain.
Domain-Invariant Features
Domain-invariant features are representations learned by a model that are robust to distribution shifts, enabling effective performance across different client data domains in federated transfer learning.
Adversarial Domain Adaptation
Adversarial domain adaptation is a technique that uses a domain discriminator trained adversarially to encourage the learning of domain-invariant feature representations, facilitating transfer across clients in federated learning.
Negative Transfer Prevention
Negative transfer prevention involves mechanisms to detect and mitigate scenarios where transferring knowledge from a source domain harms performance on the target task in a federated learning system.
Partial Parameter Transfer
Partial parameter transfer is a strategy where only a subset of a pre-trained model's layers or parameters are transferred and fine-tuned in federated learning, often freezing early feature extractors while adapting later layers.
LoRA in Federated Learning
Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning method that injects trainable low-rank matrices into a pre-trained model, enabling efficient knowledge transfer and personalization in federated learning.
Federated Self-Training
Federated self-training is a semi-supervised learning technique where a model generates pseudo-labels for unlabeled client data, which are then used as targets for further training in a decentralized manner.
Federated Representation Learning
Federated representation learning focuses on learning useful, general-purpose data embeddings from decentralized data, which can then be transferred and fine-tuned for various downstream tasks.
Federated Knowledge Distillation
Federated knowledge distillation transfers knowledge from a teacher model (or ensemble of client models) to a smaller or more efficient student model by matching outputs or intermediate representations, without sharing raw data.
Federated Neural Architecture Search
Federated neural architecture search automates the discovery of optimal model architectures for a given task directly on decentralized data, often leveraging transferable architectural patterns from source domains.
Concept Drift Adaptation
Concept drift adaptation involves techniques to detect and adjust federated models in response to changes in the underlying data distribution over time, ensuring continued performance without full retraining.
Federated Reinforcement Transfer
Federated reinforcement transfer applies transfer learning principles to decentralized reinforcement learning, allowing policies or value functions learned in a source environment to accelerate learning in target environments across multiple agents.
Federated Graph Transfer Learning
Federated graph transfer learning applies knowledge transfer techniques to graph neural networks operating on decentralized graph-structured data, such as social networks or molecular graphs.
Federated Domain Generalization
Federated domain generalization aims to learn a model from multiple source client domains that will perform well on unseen target domains, without access to target data during training.
Catastrophic Forgetting Avoidance
Catastrophic forgetting avoidance encompasses techniques, such as elastic weight consolidation or experience replay, used in continual federated learning to prevent a model from losing performance on previously learned tasks when adapting to new ones.
Transferability Estimation
Transferability estimation involves quantifying how effectively knowledge from a source model or domain can be transferred to a target federated learning task, often to guide source model selection.
Federated Multi-Source Transfer
Federated multi-source transfer learning leverages knowledge from multiple, potentially heterogeneous, source domains or models to improve learning on a target task across distributed clients.
Sim-to-Real Federated Transfer
Sim-to-real federated transfer involves training a model in a simulated source environment and adapting it to perform effectively on real-world data collected from distributed physical devices or sensors.
Vertical Federated Learning
Terms related to federated learning where different parties hold different features about the same entities. Target: Data Architects & ML Engineers.
Vertical Federated Learning (VFL)
Vertical Federated Learning (VFL) is a decentralized machine learning paradigm where different parties (clients) hold different features about the same set of entities, and collaborate to train a model without directly sharing their raw feature data.
Vertical Data Partition
A vertical data partition is a dataset split where different features (columns) of the same samples (rows) are held by different parties, forming the foundational data structure for Vertical Federated Learning.
Feature Space
In Vertical Federated Learning, the feature space refers to the complete set of input variables or attributes used to describe the entities, which is partitioned and distributed across multiple data owners.
Label Space
In Vertical Federated Learning, the label space refers to the output variable or target values, which are typically held by a single party (the label owner) while features are distributed among other parties.
Entity Alignment
Entity alignment is the process of identifying and matching the same real-world entities (e.g., customers, devices) across the vertically partitioned datasets held by different parties without revealing the full datasets.
Private Set Intersection (PSI)
Private Set Intersection (PSI) is a cryptographic protocol that allows two or more parties to compute the intersection of their private datasets (e.g., user IDs) without revealing any information about items not in the intersection, commonly used for entity alignment in VFL.
Secure Entity Resolution
Secure entity resolution is a privacy-preserving process for identifying records that refer to the same entity across multiple databases, a critical prerequisite for aligning samples in Vertical Federated Learning.
Feature Owner
A feature owner is a participant in a Vertical Federated Learning system that possesses a subset of the features (input variables) for the aligned set of entities but does not hold the labels.
Label Owner
The label owner is the participant in a Vertical Federated Learning system that holds the target values (labels) for the aligned set of entities and typically orchestrates the training process.
Split Neural Network
A split neural network is a model architecture used in Vertical Federated Learning where the network is divided into multiple parts, with each part residing on a different party (e.g., feature owners and a label owner) that holds a specific subset of the features.
Cut Layer
The cut layer is the specific layer in a split neural network where the model is divided between parties in Vertical Federated Learning, determining which computations are performed locally and which outputs are shared.
Intermediate Output
In Vertical Federated Learning, an intermediate output is the result of the forward pass up to the cut layer, which is computed by a feature owner and must be securely transmitted to the label owner (or another party) to continue the forward propagation.
Vertical Forward Propagation
Vertical forward propagation is the distributed computation of a neural network's forward pass in VFL, where each party computes its part of the model on its local features and passes intermediate results to the next party.
Vertical Backpropagation
Vertical backpropagation is the distributed computation of gradients in a split neural network during VFL, where gradients are passed backwards through the network across the participating parties to update their respective model segments.
Vertical Gradient Computation
Vertical gradient computation is the process of calculating parameter updates for a model trained with Vertical Federated Learning, which involves secure and coordinated gradient calculation across the distributed model parts.
Vertical Federated Averaging (VFA)
Vertical Federated Averaging (VFA) is a class of collaborative learning algorithms designed for the vertical data partition setting, where updates from feature owners are aggregated to form a global model.
Vertical Secure Aggregation
Vertical secure aggregation refers to cryptographic protocols used in VFL to combine model updates (e.g., gradients or intermediate outputs) from multiple feature owners without revealing any individual party's contribution.
Homomorphic Encryption for VFL
Homomorphic encryption for VFL is the application of encryption schemes that allow computations to be performed on ciphertext, enabling parties in a vertical FL system to train on encrypted intermediate results or gradients.
Vertical Multi-Party Computation (MPC)
Vertical Multi-Party Computation (MPC) refers to the use of secure multi-party computation protocols to enable joint model training over vertically partitioned data while keeping each party's input data private.
Vertical Training Protocol
A vertical training protocol is the defined sequence of communication and computation steps that coordinating parties follow to execute a complete training round in a Vertical Federated Learning system.
Vertical Inference Protocol
A vertical inference protocol defines the secure, collaborative process for making predictions using a vertically split model after training, where feature owners provide inputs without revealing them.
Privacy-Preserving Vertical FL
Privacy-preserving Vertical Federated Learning encompasses the full suite of techniques—including cryptography and differential privacy—applied to VFL to prevent data leakage during the collaborative training process.
Vertical FL with Differential Privacy
Vertical FL with Differential Privacy (DP) integrates DP mechanisms, such as adding calibrated noise to gradients or intermediate outputs, into the VFL training process to provide formal, mathematical privacy guarantees.
Federated Feature Selection
Federated feature selection is the process of identifying the most relevant features for a model from a vertically partitioned dataset across multiple owners, without centrally pooling the raw feature data.
Vertical Communication Overhead
Vertical communication overhead refers to the bandwidth cost and latency introduced by the need to exchange intermediate outputs, gradients, and encrypted messages between parties during each round of Vertical Federated Learning.
Vertical Computation Overhead
Vertical computation overhead is the additional processing cost incurred by each party in a VFL system due to cryptographic operations (e.g., encryption) and the execution of only a portion of the full model.
Vertical Federated Optimization
Vertical federated optimization involves designing and analyzing specialized optimization algorithms that account for the statistical and systems challenges unique to training models on vertically partitioned data.
Vertical Model Fairness
Vertical model fairness concerns the assessment and mitigation of biases that may arise in models trained via VFL, potentially stemming from the unequal distribution of predictive features across parties.
Vertical FL Framework
A Vertical FL framework is a software platform that provides the necessary abstractions, protocols, and tools to develop, deploy, and manage Vertical Federated Learning applications across distributed parties.
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us