Neural Architecture Search (NAS)

The search space defines the set of permissible neural network building blocks and their connectivity. For on-device RF models, this space is often constrained to depthwise separable convolutions, inverted bottlenecks, and squeeze-and-excitation blocks. A well-designed search space balances expressiveness with tractability, ensuring the controller can discover architectures that map efficiently to the target NPU or DSP instruction set without exploring invalid or uncompilable topologies.

Unlike traditional accuracy-only NAS, hardware-aware NAS directly incorporates deployment constraints into the reward function. The search algorithm optimizes a multi-objective Pareto frontier balancing TOPS/Watt, SRAM footprint, and inference latency.

• Latency feedback: Candidate architectures are profiled directly on the target accelerator (e.g., Hardware-in-the-Loop Optimization).
• Memory bounds: The search penalizes architectures exceeding the microcontroller's SRAM budget.
• Energy constraints: Models are ranked by mJ per inference, not just FLOPs.

To overcome the prohibitive computational cost of training thousands of candidates from scratch, modern NAS employs weight-sharing. A single over-parameterized supernet subsumes all possible architectures in the search space. During search, subnetworks inherit weights from the supernet, enabling rapid evaluation without full retraining. One-shot NAS trains the supernet once, then applies evolutionary or gradient-based selection to extract the optimal subnetwork for the target RF signal processing task.

DARTS relaxes the discrete architecture selection problem into a continuous optimization task. Instead of selecting a single operation, the algorithm assigns a softmax-weighted mixture of candidate operations to each edge. This allows joint optimization of network weights and architecture parameters via standard gradient descent.

• Benefit: Drastically reduces search time from thousands of GPU-days to single-digit GPU-days.
• RF Application: Enables rapid discovery of optimal IQ sample processing pipelines for specific modulation schemes.

Once a baseline architecture is discovered, compound scaling systematically expands it to meet varying resource budgets. Inspired by EfficientNet Scaling, this method uniformly scales network depth, width, and input resolution using a compound coefficient. For RF applications, this allows a single NAS-discovered cell to be scaled into a family of neural receivers optimized for different latency tiers—from a 1ms inference budget on an FPGA to a 10ms budget on a Cortex-M4.

Standard NAS often discovers architectures that perform well in FP32 but degrade severely under INT8 quantization. Quantization-aware NAS integrates simulated quantization noise directly into the supernet training and candidate evaluation loop. The search explicitly favors operations and topologies resilient to precision reduction, such as those with wider activation distributions that avoid clipping. This co-design ensures the final architecture is natively robust to Post-Training Quantization (PTQ) without requiring expensive Quantization-Aware Training (QAT) retraining.

The search space defines the set of all possible architectural building blocks the NAS algorithm can explore. For RF models, this often includes parameterized depthwise separable convolutions, inverted residual blocks, and attention mechanisms. A well-designed search space balances expressiveness with tractability, using cell-based structures where a small motif is discovered and repeated to form the full network. Macro search spaces define the outer skeleton, while micro search spaces optimize the internal operations of each cell. Constraining the space to hardware-friendly operations—such as replacing standard convolutions with shift-invariant primitives—ensures discovered architectures map efficiently to edge accelerators.

NAS is rarely a single-objective problem. Multi-objective optimization simultaneously maximizes task accuracy while minimizing hardware cost metrics such as latency, energy consumption, and peak SRAM footprint. Pareto-optimal search strategies use weighted scalarization or evolutionary algorithms to discover a frontier of architectures trading off these competing goals. For on-device RF inference, a common objective is to maximize modulation classification accuracy subject to a hard constraint of < 10 ms inference latency on a Cortex-M7 processor. Hardware-in-the-loop evaluation measures real latency on the target silicon during the search, avoiding the inaccuracies of proxy metrics like FLOPs.

Training every candidate architecture from scratch is computationally prohibitive. Weight-sharing NAS trains a single large supernet that contains all possible architectures as subgraphs. During search, candidate architectures inherit pre-trained weights from the supernet, enabling rapid evaluation without retraining. Techniques like single-path one-shot sampling ensure fair comparison by uniformly sampling paths. After the search phase, the discovered optimal subnetwork is extracted and retrained independently. This approach reduces the search cost from thousands of GPU-hours to a few GPU-days, making NAS practical for specialized RF model development.

Hardware-aware NAS incorporates a latency lookup table or a learned performance predictor directly into the search loop. Instead of optimizing for theoretical FLOPs—which correlate poorly with real latency on embedded devices—the search algorithm queries a database of pre-profiled operator timings for the target accelerator. For NPU offloading, the search space is restricted to operations supported by the accelerator's compiler. Roofline model analysis identifies whether discovered architectures are compute-bound or memory-bound, guiding the search toward balanced designs that saturate available bandwidth without exceeding SRAM limits.

DARTS relaxes the discrete architecture search problem into a continuous one by introducing architectural parameters that represent the mixing weights of candidate operations. These parameters are optimized jointly with the network weights using gradient descent. After convergence, the strongest operation per edge is selected via argmax discretization. DARTS dramatically accelerates search, but is prone to skip-connect collapse, where the search overfits to parameter-free operations. Regularization techniques like partial channel connections and robustDARTS mitigate this failure mode, ensuring the discovered architecture genuinely relies on learned feature transformations rather than identity shortcuts.

Evolutionary algorithms treat architectures as individuals in a population, applying mutation and crossover operations to generate novel topologies. A tournament selection mechanism retains high-performing candidates while discarding weak ones. For RF model optimization, mutations might alter kernel sizes, add skip connections, or replace ReLU activations with Swish functions. Evolutionary methods excel at exploring non-differentiable search spaces and naturally support multi-objective optimization via Pareto dominance ranking. The primary drawback is sample inefficiency, requiring thousands of evaluations. Aging evolution and regularized evolution improve diversity, preventing premature convergence to local optima.