Federated Transfer Learning

The core innovation of FTL is its ability to train models across parties whose data schemas have different feature dimensions and semantics. Unlike standard Federated Averaging, which requires identical feature spaces, FTL uses domain adaptation and common subspace learning to map disparate feature vectors into a shared latent representation. This is achieved by aligning the covariance matrices or using adversarial training to find a feature mapping that minimizes the discrepancy between the source and target domains.

FTL excels when one party (the source) has a fully labeled dataset and the other (the target) has only unlabeled data or a completely different label set. The architecture employs a teacher-student paradigm where soft labels (logits) or high-confidence pseudo-labels are generated by the source domain's model and securely transmitted to guide training on the target domain. This process, known as co-regularization, minimizes the empirical risk on the target domain without ever exposing raw source labels.

FTL protocols are built on a privacy-preserving computation layer that combines multiple cryptographic primitives. The process typically involves:

• Additive secret sharing to split intermediate model parameters between parties.
• Homomorphic encryption to compute loss functions and gradients directly on encrypted data.
• A neutral third-party server that aggregates encrypted updates without accessing plaintext. This ensures that neither party can reconstruct the other's proprietary feature space or label distribution from the exchanged gradients or activations.

To bridge the gap between non-overlapping feature spaces, FTL learns a shared representation using techniques like Kernel Canonical Correlation Analysis (KCCA) or neural alignment layers. Both parties project their local data into a low-dimensional common subspace where the statistical distributions are aligned. The objective is to maximize the correlation between the projected views while preserving the discriminative structure needed for the target task, effectively creating a virtual common feature space for collaborative learning.

The FTL training loop optimizes a composite loss function that balances two competing goals:

1. Alignment Loss: Minimizes the distance between the source and target distributions in the common subspace (e.g., using Maximum Mean Discrepancy or adversarial loss).
2. Task-Specific Loss: Minimizes the prediction error on the labeled source data while propagating useful gradients to the target model. This dual-objective prevents negative transfer, where forced alignment destroys useful discriminative features in the target domain.

FTL is the natural extension of Vertical Federated Learning (VFL) for scenarios where entities hold different features for the same set of users but also require domain adaptation. In this setting, an entity resolution step first identifies overlapping samples using Private Set Intersection (PSI). Then, instead of assuming identical label distributions, FTL's transfer component handles the case where the label semantics or granularity differ between the two vertical partitions, enabling collaboration between, for example, a bank and an insurer with different risk classification schemas.

A distributed learning architecture where a deep neural network is partitioned between a client and a server. The client processes initial layers and transmits only intermediate activations (smashed data) rather than raw data. This is a core component of Federated Transfer Learning, as it allows parties to keep their base feature extractors private while collaboratively training a shared top model on the server side, naturally handling feature space disparity.

The fundamental challenge in federated systems arising from Non-IID data distributions across clients. In Federated Transfer Learning, this is the explicit problem to be solved:

• Feature space heterogeneity: Different parties observe different attributes (e.g., one bank uses income, another uses postal code).
• Label space heterogeneity: Parties may have different classification tasks.
• Sample space heterogeneity: Very little overlap in the user base between parties. Standard Federated Averaging fails here, necessitating transfer learning techniques.

A cryptographic protocol that allows a central server to compute the sum of encrypted model updates from multiple clients without being able to inspect any individual client's contribution in plaintext. In a Federated Transfer Learning context, secure aggregation protects the gradients of the shared top model during collaborative training, ensuring that even the aggregated knowledge does not leak proprietary feature representations from any single party.

A transfer learning sub-field focused on adapting a model trained on a source domain to perform well on a different but related target domain. Federated Transfer Learning generalizes this concept to a multi-party, privacy-preserving setting. Techniques like adversarial domain alignment are used to learn a common feature representation across different clients' data distributions without directly sharing the underlying data, effectively minimizing the domain shift in a federated manner.

A communication-efficient alternative to sharing model weights. Clients exchange model outputs (logits) on a public, unlabeled dataset instead of gradients. This is highly synergistic with Federated Transfer Learning, especially when clients have heterogeneous model architectures. Knowledge distillation can transfer the predictive behavior of a private teacher ensemble into a global student model, bypassing the need for co-architectural alignment required by standard weight averaging.

An encryption scheme that permits computation directly on ciphertexts. In Federated Transfer Learning, this allows the central server to aggregate encrypted model updates from multiple parties without ever decrypting them. This provides a stronger privacy guarantee than differential privacy for the aggregation step, ensuring that the server learns only the final encrypted model and nothing about individual contributions, which is critical for highly regulated cross-silo collaborations.