Federated Neural Architecture Search (FNAS)

FNAS distributes the architecture search process across clients, where each client explores a subset of the search space using its local data. Controllers (often recurrent neural networks or evolutionary algorithms) propose candidate architectures, which are trained locally and evaluated without centralizing raw data. Only architecture performance metrics and lightweight model descriptors are transmitted back to the server, preserving data locality while enabling collaborative topology discovery.

Unlike manual designs that assume IID data, FNAS explicitly searches for architectures that are robust to statistical heterogeneity. The search objective incorporates client-specific performance variance, penalizing topologies that overfit to dominant data distributions. This yields architectures with inherent personalization capacity—layers that learn shared representations while allowing client-specific branches to adapt to local distribution shifts without manual intervention.

FNAS employs weight-sharing supernets to amortize search costs. Instead of training thousands of candidate architectures independently, a single over-parameterized supernet encodes all possible sub-networks. Clients sample and evaluate sub-networks from this shared supernet, drastically reducing communication rounds. Only gradient updates to the supernet are aggregated, achieving architecture search with communication overhead comparable to standard federated training.

FNAS frameworks jointly optimize for accuracy, model size, inference latency, and communication cost. The search algorithm discovers Pareto-optimal architectures that balance these competing objectives across heterogeneous client hardware. This is critical for cross-device deployments where edge nodes range from powerful GPU servers to resource-constrained medical IoT devices, each requiring tailored compute profiles.

FNAS can incorporate differential privacy guarantees directly into the architecture search loop. By clipping and noising the gradients used to update the controller, the search process itself becomes privacy-preserving. This prevents architecture selection from leaking sensitive information about local data distributions through the chosen topology, a critical requirement for clinical environments governed by HIPAA and GDPR.

FNAS supports two primary search paradigms:

• Evolutionary algorithms: Maintain a population of architectures, applying mutation and crossover operations across clients to evolve topologies over generations
• Gradient-based methods: Use differentiable architecture parameters that are jointly optimized with model weights via bi-level optimization Each approach offers distinct trade-offs in search efficiency and architectural diversity.

The foundational automated machine learning technique that FNAS extends into the federated domain. NAS uses a search strategy (reinforcement learning, evolutionary algorithms, or gradient-based methods) to explore a predefined search space of possible layer types and connections. A performance estimation strategy evaluates candidate architectures without full training. FNAS distributes this search across clients, evaluating architectures on local data distributions to find topologies inherently robust to heterogeneity.

A critical privacy guarantee integrated into FNAS pipelines. Architecture search signals — validation accuracy, latency, or gradient information — can leak sensitive properties of local datasets through membership inference or model inversion attacks. Differentially private FNAS adds calibrated noise to architecture performance reports or applies gradient clipping with noise injection during the search phase, ensuring that discovered architectures reveal no individual patient-level information.