Cross-Silo FL

Unlike Cross-Device FL with millions of ephemeral devices, Cross-Silo FL operates with a small number (e.g., 2-100) of known, reliable organizational participants. These entities, such as hospitals or financial institutions, have:

• Stable, high-bandwidth connectivity to the central aggregator.
• Formal participation agreements and Service Level Agreements (SLAs).
• Significant computational resources (e.g., data center GPUs) for local training. This stability allows for more complex training protocols and reduces the system heterogeneity challenges prevalent in cross-device scenarios.

Cross-Silo FL typically assumes a horizontal (sample-based) data partition. Each organization (or 'silo') holds a different set of data samples (e.g., patient records, financial transactions) that share the same feature space. For example:

• Hospital A and Hospital B both record the same clinical features (blood pressure, lab results) for their respective, non-overlapping patient populations.
• The goal is to train a model that generalizes across the union of samples from all silos without centralizing the sensitive raw data. This contrasts with Vertical FL, where different parties hold different features for the same entities.

The primary driver for Cross-Silo FL is compliance with stringent data governance regulations (e.g., GDPR, HIPAA, GLBA) that prohibit the centralization of sensitive data. Privacy preservation is non-negotiable and is enforced through a multi-layered technical stack:

• Cryptographic Protocols: Secure Aggregation ensures the server only sees the sum of client updates, not individual contributions. Homomorphic Encryption allows computation on encrypted model updates.
• Formal Privacy Guarantees: Differential Privacy (DP) adds calibrated noise to updates to mathematically bound privacy loss.
• Trust Models: Assumptions range from a honest-but-curious server to fully Byzantine-robust protocols, depending on the consortium's trust dynamics.

Data across silos is almost never Independent and Identically Distributed (IID). This statistical heterogeneity is a defining challenge:

• Feature Distribution Skew: The prevalence of certain conditions or transaction types varies per institution.
• Label Distribution Skew: One hospital may specialize in cardiology, another in oncology.
• Concept Drift: The same label (e.g., 'fraud') may have subtly different underlying patterns in different banks. This heterogeneity causes client drift, where local models diverge, hindering global convergence. Algorithms like FedProx and SCAFFOLD are specifically designed to mitigate this.

While communication efficiency is still a concern, the primary optimization goal is often final model accuracy and robustness, not minimizing bytes transmitted. This is due to the stable, high-bandwidth environment. Key algorithmic considerations include:

• Multiple Local Epochs: Clients perform many passes over their local data per communication round, leading to significant Local SGD.
• Sophisticated Aggregation: Use of advanced federated optimization techniques beyond simple Federated Averaging (FedAvg), such as adaptive server optimizers or techniques to correct for client drift.
• Robust Aggregation: Methods like median-based or clipped-mean aggregation are used to ensure Byzantine robustness against potentially malicious or faulty updates from a small number of silos.

Cross-Silo FL is deployed in industries where data is highly valuable, sensitive, and regulated. Real-world examples include:

• Healthcare: Multiple hospitals collaboratively training a diagnostic model for rare diseases without sharing patient records. This is a core application of Healthcare Federated Learning.
• Finance: Banks collaborating to build a better anti-money laundering (AML) or fraud detection model without exposing proprietary transaction data.
• Manufacturing: Different factories within a corporation improving a predictive maintenance model using their local operational data, which may be competitively sensitive.
• Pharmaceuticals: Drug discovery collaborations between research institutions where molecular assay data is proprietary.

Allows telecom operators in different regions or countries to improve network performance models (e.g., for radio resource management or predictive maintenance) by learning from each other's network telemetry. Proprietary network configuration and customer usage data is not exchanged.

• Example: Operators collaboratively training a model to predict cell tower failures.
• Key Driver: Competitive advantage and regulations governing telecommunications data localization.

The foundational aggregation algorithm for federated learning. The central server computes a weighted average of model updates (e.g., weight deltas) received from participating clients to form a new global model. In Cross-Silo FL, weights are often based on each organization's dataset size.

• Core Mechanism: Server aggregates client model parameters: w_global = Σ (n_k / N) * w_k
• Cross-Silo Application: Organizations (silos) are reliable, allowing for more local epochs and fewer communication rounds compared to cross-device FL.

A paradigm where different organizations hold different feature sets for the same set of entities (e.g., a bank has financial data and a retailer has purchase history for the same customers). VFL enables collaborative model training without sharing raw vertical data partitions.

• Contrast with Cross-Silo FL: Cross-Silo is typically horizontal FL (same features, different samples). VFL is feature-partitioned.
• Use Case: Joint credit scoring model between a bank and an e-commerce platform.

A cryptographic protocol that allows a server to compute the sum of client model updates without being able to inspect any individual client's contribution. This protects data privacy even from the central coordinator.

• Privacy Guarantee: The server learns only the aggregated model update, not individual gradients or weights.
• Critical for Cross-Silo: Essential when silos (e.g., competing hospitals) require guarantees that their proprietary updates cannot be reverse-engineered.

The fundamental challenge where local data distributions across clients are not independent and identically distributed (Non-IID). In Cross-Silo FL, each organization's data can have vastly different statistical properties.

• Impact: Causes client drift, where local models diverge from the global objective, slowing convergence and harming final accuracy.
• Mitigation: Algorithms like FedProx and SCAFFOLD are designed to correct for this drift.

A rigorous mathematical framework for quantifying and bounding privacy loss. In FL, DP-SGD can be applied locally by clients, who add calibrated noise to their updates before sending them to the server for aggregation.

• Formal Guarantee: Provides an (ε, δ)-differential privacy guarantee, making it statistically unlikely to determine if any individual's data was in the training set.
• Cross-Silo Use: Often applied in healthcare or finance FL to provide a robust, auditable privacy guarantee atop secure aggregation.

An alternative distributed learning technique where a neural network is vertically split between a client and a server. The client computes the initial layers and sends the intermediate activations (called smashed data) to the server, which completes the forward and backward pass.

• Comparison to FL: Reduces client compute load but requires continuous, secure communication of intermediate data during training.
• Cross-Silo Context: Can be used when one party has the labels and significant compute, while others have feature data but limited resources.