Federated Prompt Tuning

Instead of crafting discrete natural language instructions, Federated Prompt Tuning prepends a sequence of continuous, trainable embedding vectors to the model input. These 'soft prompts' are optimized through backpropagation to condition the frozen foundation model for specific clinical tasks.

• Mechanism: The prompt tokens are virtual—they exist in the embedding space and are initialized randomly or from a task-specific vocabulary embedding.
• Efficiency: Only a few thousand parameters are trained per task, compared to billions for full fine-tuning.
• Example: A soft prompt of length 20 tokens (approximately 20 × 4096 = ~82K parameters for a 7B model) can steer a frozen LLM to perform radiology report summarization.

Each participating hospital trains its local soft prompt on private clinical data. Only these lightweight prompt vectors are transmitted to a central server, where they are securely aggregated using algorithms like Federated Averaging (FedAvg) to produce a global consensus prompt.

• Privacy Guarantee: Raw patient data, model weights, and gradients never leave the institution.
• Communication Cost: Transmitting a prompt update is orders of magnitude smaller than sharing full model gradients (kilobytes vs. gigabytes).
• Aggregation Strategy: The server performs a weighted average of local prompts, where weights can be proportional to the size of each institution's local dataset.

The base large language model remains completely frozen across all institutions. No weight updates are applied to the transformer layers, attention mechanisms, or feed-forward networks.

• Stability: Prevents catastrophic forgetting of the model's general medical knowledge acquired during pre-training.
• Multi-Task Serving: A single frozen model instance can serve multiple clinical tasks simultaneously by simply swapping the prepended soft prompt.
• Hardware Efficiency: Institutions do not need to compute or store full model gradients, enabling participation from sites with limited GPU memory.

Federated Prompt Tuning is ideally suited for cross-silo healthcare networks where a moderate number of reliable institutions (e.g., 5-50 hospitals) collaborate to adapt a shared model to specialized clinical vernacular.

• Domain Alignment: The global prompt learns to condition the model on institution-agnostic clinical patterns while local training captures site-specific terminology.
• Use Case: Adapting a general medical LLM to generate structured oncology reports by learning a shared prompt from tumor board notes across five cancer centers.
• Personalization: A global prompt can be further fine-tuned locally for a few additional steps to create a personalized prompt for a specific hospital's documentation style.

The number of trainable virtual tokens in the soft prompt directly controls the capacity of the adaptation. Longer prompts can capture more complex task semantics but increase communication overhead.

• Short Prompts (5-20 tokens): Suitable for simple classification or entity extraction tasks. Minimal bandwidth footprint.
• Long Prompts (50-100 tokens): Required for complex generative tasks like clinical note synthesis or multi-step reasoning.
• Optimization: Techniques like prompt decomposition or sparse prompting can be applied to maintain expressivity while reducing the effective parameter count.

To provide formal privacy guarantees, local prompt updates can be clipped and noised before transmission using differential privacy mechanisms. This bounds the influence of any single patient record on the final global prompt.

• DP-SGD for Prompts: Gaussian noise is added to the prompt gradients during local training, and the total gradient norm is clipped.
• Privacy Budget Accounting: The privacy loss (ε, δ) is tracked across communication rounds, allowing institutions to set a maximum privacy budget.
• Utility-Privacy Balance: Longer prompts and more communication rounds improve task accuracy but consume the privacy budget faster, requiring careful calibration.

A decentralized training paradigm where only a small subset of adapter parameters are shared and aggregated across institutions to adapt a frozen foundation model to clinical tasks. Unlike full fine-tuning, PEFT methods like prompt tuning and LoRA minimize communication overhead and preserve data locality. The central server never accesses raw patient data, only the lightweight adapter updates. This approach is critical for healthcare NLP where bandwidth is limited and privacy is paramount.

A specific federated PEFT method that trains and aggregates low-rank decomposition matrices injected into a foundation model's layers. Each institution trains a pair of small matrices (A and B) whose product represents a weight update, keeping the original model frozen. Only these low-rank matrices are transmitted and aggregated via Federated Averaging. This enables efficient cross-institutional adaptation for clinical entity extraction and medical coding without sharing the base model weights.

The process of collaboratively fine-tuning a foundation model across distributed healthcare datasets using formatted instruction-output pairs. Institutions contribute local examples of clinical directives and their correct structured outputs—such as 'Summarize this discharge note' paired with a gold-standard summary. The aggregated updates enable the model to better follow clinical commands and generate structured medical reports, all without centralizing the underlying patient instructions.

A privacy-preserving architecture where a central language model queries distributed local vector stores at each institution to retrieve relevant clinical context. When a query is received, the model dispatches retrieval requests to each hospital's secure vector database, which returns relevant document chunks without exposing raw patient data. The model then synthesizes a grounded answer from the aggregated, de-identified context. This enables evidence-based clinical decision support across institutional boundaries.

A model compression technique where a global teacher model's knowledge is transferred to smaller student models at each site. Instead of sharing private model gradients, institutions only exchange the teacher's output logits on a public or synthetic reference dataset. Each local student model is trained to mimic the teacher's softened probability distributions. This avoids exposing patient data while enabling resource-constrained hospital edge devices to benefit from the collective intelligence of the federated network.

A decentralized prompt engineering technique where institutions collaboratively refine multi-step reasoning prompts. Each site tests and improves structured reasoning templates—such as 'Let's think step by step about this patient's symptoms'—on their local clinical cases. The most effective prompt structures are aggregated centrally, enabling a shared language model to generate more accurate and explainable clinical diagnoses by breaking down complex cases into logical reasoning steps without sharing the cases themselves.