Once-For-All (OFA)

The core of OFA is a single, over-parameterized supernet that subsumes a vast search space of potential sub-networks (subnets). This search space is defined along multiple dimensions:

• Depth: Number of layers/blocks.
• Width: Number of channels per layer.
• Kernel Size: Choice of convolution kernel (e.g., 3x3, 5x5, 7x7).
• Expand Ratios: Width multipliers within mobile inverted bottleneck blocks. All sub-networks share the supernet's weights, enabling their performance to be estimated without individual training.

OFA employs a novel training algorithm called Progressive Shrinking to prevent the catastrophic interference that occurs when training a supernet with vastly different subnets. The training proceeds in stages:

1. Train the largest possible subnet (full supernet) to convergence.
2. Fine-tune this network to support smaller subnets by progressively adding smaller depth, width, and kernel size options.
3. Each stage inherits knowledge from the previous, larger stage, ensuring all subnets achieve high accuracy. This method is far more efficient than training each architecture from scratch.

After the supernet is trained, the search for an optimal subnet is a zero-cost process. Given a target hardware platform (e.g., a specific microcontroller) and constraints (latency, memory), the search involves:

• Using a pre-characterized hardware cost model (e.g., a latency lookup table) to estimate the performance of any subnet.
• Applying an evolutionary search or other optimization algorithm to find the subnet that maximizes accuracy while satisfying the hardware constraints. No further training or fine-tuning is required; the subnet's weights are directly inherited from the supernet.

OFA's fundamental innovation is the complete decoupling of the training and search phases.

• Training Phase (Once): The supernet is trained via Progressive Shrinking. This is computationally expensive but done only once.
• Search Phase (For All): For each new hardware target or constraint, an efficient search finds the best subnet. This phase is fast and requires no backpropagation or GPU time. This paradigm enables rapid customization for diverse edge devices from a single trained model, eliminating the need for per-device retraining.

OFA is particularly effective for TinyML and microcontroller deployment, where constraints are severe. It can discover subnets optimized for:

• Kilobyte-scale memory footprints (e.g., < 500KB RAM/Flash).
• Milliwatt-level power budgets and strict latency limits.
• Diverse hardware backends (CPU, GPU, NPU, MCU). By searching within the trained supernet, it finds architectures that manually designed models or standard NAS might miss, pushing the Pareto frontier of accuracy vs. efficiency for embedded systems.

OFA addresses key limitations of earlier Neural Architecture Search methods:

• vs. Reinforcement Learning (RL) NAS: OFA replaces the expensive process of training thousands of candidate models from scratch with a single supernet training run.
• vs. Differentiable NAS (e.g., DARTS): OFA provides a more stable training process (Progressive Shrinking) and delivers a standalone, deployable model for each constraint, not just a cell structure.
• General Efficiency: The 'train once, search for many' approach makes OFA highly scalable for production environments supporting a heterogeneous fleet of edge devices.

OFA is a cornerstone for deploying machine learning on microcontrollers and other edge devices. By training a single supernet, developers can extract sub-networks that precisely match the memory, latency, and power constraints of a target MCU (e.g., ARM Cortex-M series). This eliminates the need to design and train a unique model for every hardware variant in a product line.

• Key Benefit: A single OFA supernet can service an entire product family, from a high-end gateway to a low-power sensor node.
• Example: Extracting a 50KB model for a battery-powered environmental sensor and a 200KB model for a smart camera from the same supernet.

The OFA paradigm directly addresses the challenge of heterogeneous hardware ecosystems. The supernet is designed with scalable dimensions—depth (number of layers), width (number of channels), and kernel size—allowing for dynamic adaptation.

• Dynamic Adaptation: For a given hardware platform, a search algorithm (or a simple lookup) selects the optimal sub-network configuration (depth, width, kernel size) that maximizes accuracy within the hardware's latency or memory budget.
• Use Case: A smartphone app can deploy a larger, more accurate sub-network when the device is plugged in and switch to a tiny, efficient sub-network when on battery power, all from the same supernet.

OFA provides a scalable framework for managing machine learning models across massive IoT fleets. Instead of storing and deploying thousands of individually trained models, a single supernet weight file is distributed. Each device then runs a lightweight sub-network extraction process based on its unique hardware fingerprint or performance requirements.

• Operational Efficiency: Dramatically simplifies model versioning, updates, and rollbacks. Updating the central supernet can improve performance for all derived sub-networks.
• Example: A manufacturer of industrial vibration sensors can deploy one OFA supernet to all units globally. Units with more powerful processors automatically extract a more complex model for finer anomaly detection.

OFA is a foundational technique for One-Shot NAS. By training the supernet once, it creates a performance estimation network that allows for the rapid evaluation of millions of candidate architectures without any further training.

• Search Process: After supernet training, a search algorithm (e.g., evolutionary search) samples sub-networks. Their accuracy is estimated via weight sharing, and their latency is predicted using a hardware cost model.
• Result: Finding a Pareto-optimal model for a new hardware target takes orders of magnitude less time and compute than training-based NAS methods.

The OFA supernet serves as a standardized benchmarking and research vehicle for the hardware-aware NAS community. It allows researchers to fairly compare new search algorithms, pruning techniques, or quantization methods on a consistent, pre-trained foundation.

• Consistent Baseline: New search strategies or performance estimators can be evaluated by how well they find high-performing sub-networks within the OFA search space.
• Facilitates Innovation: Researchers can focus on improving the search or extraction process without the prohibitive cost of supernet training.

While the core OFA training is done in the cloud, the paradigm supports downstream federated learning or few-shot adaptation on devices. The lightweight sub-networks extracted from the supernet can serve as excellent starting points for further personalization using local user data.

• Privacy & Efficiency: A base sub-network is extracted for the device's hardware. This model can then be fine-tuned locally via federated averaging or a few gradient steps, adapting to local patterns without exposing raw data.
• Example: A keyword spotting model on smart home devices could start from a generic OFA sub-network and then personalize to a specific user's accent and vocabulary on-device.

One-Shot NAS is the overarching family of methods to which OFA belongs. It trains a single, over-parameterized supernet one time. The key innovation is weight sharing, where all possible sub-networks (child models) inherit their parameters from this supernet. This allows for the evaluation of thousands of architectures by simply sampling different paths through the supernet, eliminating the need for costly individual training from scratch. OFA is a specific, highly scalable implementation of the one-shot paradigm.

A supernet is the central construct in One-Shot NAS and OFA. It is a large, directed acyclic graph that encapsulates the entire search space. Each node represents a layer, and edges represent possible operations (e.g., 3x3 conv, 5x5 conv, skip connection). During supernet training, all paths are optimized simultaneously. For inference, a sub-network is extracted by selecting a single path. The quality of the supernet's training and its weight-sharing scheme directly determines the accuracy of the extracted sub-networks.

Hardware-Aware NAS extends traditional NAS by incorporating deployment constraints directly into the search objective. Instead of just maximizing accuracy, it optimizes for metrics like:

• Latency (inference time on target MCU)
• Peak Memory Usage (RAM/Flash)
• Energy Consumption (microjoules per inference) OFA is a hardware-aware method because its search space includes dimensions for depth, width, and kernel size, allowing it to extract sub-networks that meet specific hardware budgets without retraining.

In NAS, the search space defines the universe of all possible architectures the algorithm can discover. OFA's search space is particularly flexible, defined along three axes:

• Depth: The number of layers in the network.
• Width: The number of channels (neurons) in each layer.
• Kernel Size: The spatial dimensions of convolutional filters (e.g., 3, 5, 7). The supernet is constructed to contain all valid combinations. A good search space balances expressiveness (ability to find high-performing models) with size (to make search tractable).

Designing for microcontrollers is inherently a multi-objective optimization problem. Engineers must balance:

• Model Accuracy (primary task performance)
• Inference Latency (real-time operation)
• Memory Footprint (fitting within KB of RAM/Flash)
• Energy Efficiency (battery life) OFA addresses this by constructing a Pareto frontier of models—a set of optimal trade-off solutions where improving one metric worsens another. The search process in OFA can be guided to find sub-networks at different points on this frontier.

A performance estimator is any method used to predict the final accuracy or hardware cost of a candidate architecture without full training. In OFA, the supernet itself acts as a highly efficient performance estimator via weight sharing. For hardware metrics, OFA often employs a hardware cost model—a small, pre-built lookup table or neural network that predicts latency, memory, or energy for a given sub-network configuration on the target device. This allows for rapid evaluation of thousands of candidates during the search phase.