Cross-Silo Federated Learning

Clients are reliable organizations like hospitals, banks, or research labs, not consumer devices. This fundamentally changes the trust and communication model.

• Participants: A small, known number of entities (e.g., 2-50).
• Reliability: Clients are generally stable, with high availability and reliable network connections.
• Stakes: Data is highly sensitive (PHI, financial records, trade secrets), and model failures have significant real-world consequences.

Each client holds a large, deep dataset covering many samples but potentially with different feature sets, known as vertical partitioning.

• Data Volume: Individual datasets are large, often containing thousands to millions of records.
• Non-IID by Design: Data distributions differ meaningfully between silos due to institutional specialization (e.g., one hospital's patient demographics vs. another's).
• Feature Disparity: Different silos may collect different types of data (features) on overlapping entities, making alignment a key challenge.

Privacy guarantees must be cryptographically or mathematically verifiable, not just procedural. Trust is limited even among participants.

• Core Techniques: Relies on Differential Privacy, Secure Multi-Party Computation (MPC), and Homomorphic Encryption.
• Privacy Budget Management: Strict tracking of the privacy budget (epsilon ε) is required across training rounds.
• Regulatory Driver: Compliance with regulations like HIPAA, GDPR, or financial industry rules is a primary motivator, necessitating auditable privacy proofs.

While clients are reliable, the size of model updates and the cryptographic overhead of privacy techniques become the primary bottlenecks, not client availability.

• Update Size: Models are often large (e.g., deep neural networks for medical imaging), leading to significant bandwidth use per round.
• Crypto Overhead: Protocols like Secure Aggregation or Fully Homomorphic Encryption add substantial computational and communication costs.
• Optimization Focus: Techniques like gradient compression and sparsification are critical to make training feasible.

A central server (aggregator) coordinates training but is not necessarily trusted with raw data or individual model updates. The trust model is explicit and often adversarial.

• Server Role: Orchestrates rounds, performs secure aggregation, and distributes the global model.
• Byzantine-Robust Aggregation: The aggregation rule (e.g., Krum, Median) must be resilient to model poisoning attacks from potentially malicious participants.
• Trusted Execution Environments (TEEs): Hardware like Intel SGX may be used to create a trusted enclave for the aggregation server.

The goal is often not a single global model, but a family of models tailored to institutional or demographic groups, addressing data heterogeneity and fairness.

• Personalized Federated Learning: Techniques like FedAvg are adapted to produce client-specific models that perform well on local data distributions.
• Bias Mitigation: Explicit strategies are needed to ensure the global model does not become biased towards the data distribution of the largest or most influential silo.
• Evaluation Challenge: Model performance must be evaluated across all silos to ensure generalized utility.

Banks and financial institutions can build more robust fraud detection and anti-money laundering models by learning from transaction patterns across the entire consortium. Since raw transaction data is highly sensitive and cannot leave institutional firewalls, cross-silo FL allows for a global model that understands emerging fraud patterns seen by any single bank, improving security for all participants.

Pharmaceutical companies can accelerate drug discovery by collaboratively training models on proprietary molecular and clinical trial data. This is a key application of Molecular Informatics and Bio-AI. Cross-silo FL allows for training on larger, more diverse datasets to predict drug efficacy or protein-ligand binding, while preserving the intellectual property and patient privacy of each company's dataset.

Mobile network operators (MNOs) can use cross-silo FL to optimize Radio Access Network parameters like handover thresholds and beamforming without sharing proprietary network performance data. By training on data from multiple operators, the model learns to improve overall network energy efficiency and quality of service in diverse real-world conditions, a core goal of AI-Enhanced RAN.

A cryptographic protocol fundamental to federated learning that allows a central server to compute the sum of model updates from multiple clients without being able to inspect any individual client's contribution. This prevents the server from performing model inversion attacks to reconstruct private training data from a single update.

• Mechanism: Often uses masking with secret shares where clients add a random mask to their update. The masks are structured to cancel out when all updates are summed.
• Guarantee: Provides input privacy against an honest-but-curious central server.
• Challenge: Requires reliable client participation; if a client drops out, their mask must be reconstructed to unmask the aggregate.

A rigorous mathematical framework for quantifying and bounding the privacy loss of an individual when their data is used in a computation. In Cross-Silo FL, it is often applied at the client level by adding calibrated noise to local model updates before they are sent to the server.

• Privacy Budget (ε): A parameter controlling the strength of the guarantee. A smaller ε means stronger privacy but lower model utility.
• Gaussian/Laplace Mechanism: The standard algorithms for adding noise. The noise scale is proportional to the sensitivity of the function (e.g., the maximum influence a single client's dataset can have on the model update).
• Composition: The total privacy budget ε is consumed over multiple training rounds, requiring careful accounting.

A class of aggregation rules designed to maintain the integrity of the global model when a fraction of clients are malicious or faulty (Byzantine clients). These clients may send arbitrary, adversarial updates to poison the model.

• Common Algorithms: Krum, Multi-Krum, Median, and Trimmed Mean. These algorithms identify and reject statistical outliers among the client updates.
• Trade-off: Robustness often comes at the cost of slower convergence, as some legitimate updates from non-IID data may also be filtered out.
• Critical for Cross-Silo: While silos are more reliable than mobile devices, the high value of their data makes them attractive targets for sophisticated model poisoning attacks.

A form of encryption that allows computation on ciphertexts. When decrypted, the result matches the outcome of operations performed on the plaintext. In FL, it can be used to perform secure aggregation where the server operates on encrypted model updates.

• Partial vs. Fully Homomorphic (FHE): Partial HE (e.g., Paillier) supports only addition, which is sufficient for averaging updates. FHE supports arbitrary circuits (addition and multiplication) but is computationally prohibitive for large neural networks.
• Use Case: Provides a stronger cryptographic guarantee than Secure Aggregation alone, protecting against a malicious server that may deviate from the protocol.
• Overhead: Introduces significant computational and communication overhead, making it challenging for large models but potentially viable for smaller updates or summary statistics.

An approach that addresses data heterogeneity (non-IID data) across silos by learning tailored models for each client or groups of similar clients, rather than a single global model. This is highly relevant in Cross-Silo FL where each hospital or bank has a unique data distribution.

• Methods: Include Local Fine-Tuning (clients adapt the global model locally), Multi-Task Learning (learning separate but related models), and Model Interpolation (mixing global and local models).
• Meta-Learning: Techniques like Per-FedAvg use meta-learning to find a global model initialization that can be quickly personalized by each client with few local steps.
• Goal: Achieve high performance on each client's local task while still benefiting from the collective knowledge of the federation.

The application of federated computation principles to data analysis tasks beyond model training. It allows institutions to compute aggregate statistics (e.g., averages, histograms, counts) over their joint data without sharing the raw records.

• Examples: A group of hospitals could compute the average length of patient stay for a specific diagnosis, or the frequency distribution of lab test values across their combined populations.
• Techniques: Relies heavily on Secure Aggregation and Differential Privacy to protect individual data points within each silo's contribution to the aggregate.
• Relationship to FL: Often serves as a precursor or complementary activity to Cross-Silo FL, helping organizations understand data distributions and feasibility before embarking on joint model training.