Federated Neural Architecture Search

Unlike centralized NAS that requires a monolithic dataset, Federated NAS executes the search algorithm locally on each client. A controller network proposes candidate architectures, which are trained on local Non-IID Data and evaluated without the data ever leaving the edge device. Only the architecture encodings and performance metrics are transmitted back to the aggregation server, ensuring raw proprietary data remains air-gapped.

Federated NAS operates on two distinct optimization loops:

• Outer Loop (Architecture Search): A meta-controller or evolutionary algorithm explores the discrete search space of possible layer types and connections.
• Inner Loop (Weight Optimization): Candidate architectures are trained locally using Federated Averaging (FedAvg) or FedProx to converge on optimal weights. This bilevel structure decouples model topology discovery from standard parameter training.

The search process can be constrained by the heterogeneous compute profiles of the federated clients. A factory-floor edge device with a Neural Processing Unit (NPU) imposes different latency and memory budgets than a cloud GPU. Federated NAS frameworks incorporate these hardware constraints directly into the reward function, ensuring the discovered architecture is not just accurate but also deployable on the target TinyML or edge silicon without manual pruning.

To prevent Model Inversion Attacks that could reconstruct sensitive factory telemetry from shared architecture gradients, Federated NAS often integrates Differential Privacy mechanisms. By clipping gradient norms and injecting calibrated Gaussian noise into the architecture parameter updates, the system provides a mathematically provable privacy guarantee. This ensures that the optimal neural topology is discovered without leaking the statistical signatures of rare production defects.

To avoid training thousands of candidate architectures from scratch—a prohibitive cost in federated settings—Federated NAS often employs a weight-sharing supernet. This is a single, large directed acyclic graph that encapsulates all possible sub-networks. Local clients train the supernet once, and the search algorithm samples and evaluates sub-networks by inheriting weights directly, reducing the search cost from thousands of GPU-hours to a single round of federated training.

In a Cross-Silo Federated Learning topology, where a small number of reliable factories participate, the aggregation server can employ secure protocols to merge architecture proposals. Using Secure Aggregation, the server computes the sum of encrypted architecture encodings, preventing any single party—including the server—from inspecting another factory's discovered topology. This is critical for competitive manufacturing environments where model architecture itself is considered intellectual property.