Neural Architecture Search (NAS)

The search space defines the set of all possible architectural building blocks the algorithm can assemble. For RF inference, this typically includes parameterized operations like depthwise separable convolutions, dilated convolutions with varying kernel sizes, and squeeze-and-excitation blocks. A well-constrained search space that leverages domain knowledge—such as the need for 1D convolutions over IQ samples—dramatically reduces the search cost and prevents the discovery of architectures that are invalid for the target FPGA fabric.

Unlike traditional NAS that optimizes solely for accuracy, hardware-aware NAS incorporates direct feedback from the target FPGA into the search objective. A latency lookup table or a roofline model is used to estimate the on-device performance of each candidate operation. The search algorithm then solves a multi-objective optimization problem, discovering architectures that sit on the Pareto frontier of classification accuracy versus inference latency. This prevents the selection of a high-accuracy topology that cannot meet real-time streaming constraints.

Training every candidate architecture from scratch is computationally prohibitive. One-shot NAS methods construct a single, over-parameterized supernet that contains every possible path in the search space as a subgraph. During search, all sub-networks share the supernet's weights. The search algorithm then evaluates candidates by inheriting these shared weights without retraining, reducing the search time from thousands of GPU-hours to a single training run. The final discovered sub-network is then retrained in isolation to achieve peak accuracy.

The controller or evolutionary algorithm is guided by a composite reward function that balances competing objectives. A typical formulation for edge RF deployment is:

• Accuracy: Maximize modulation classification performance on a held-out validation set.
• Latency: Minimize the end-to-end inference time in milliseconds on the target FPGA.
• Model Size: Minimize the number of parameters to fit within on-chip Block RAM (BRAM) constraints. This scalarized reward allows the search to automatically penalize architectures that exceed the resource budget of a specific Xilinx or Intel device.

The search is driven by a controller that proposes new architectures. Evolutionary algorithms mutate the best-performing architectures from a population, replacing operations or adding skip connections. Reinforcement learning controllers use a recurrent neural network to sequentially sample architectural decisions, treating validation accuracy as the reward signal. For hardware-aware NAS, evolutionary methods are often preferred as they naturally handle the discrete, constrained optimization landscape of FPGA-compatible operations.

DARTS relaxes the discrete search space into a continuous one by placing a mixture of candidate operations on each edge, weighted by architectural parameters. This allows both the network weights and the architecture parameters to be optimized jointly via gradient descent. After convergence, the strongest operation is selected. While computationally efficient, DARTS requires careful regularization to prevent skip-connection collapse, where the search greedily selects parameter-free operations that converge faster but yield poor final accuracy for complex RF signal classification.

A specialized NAS paradigm that incorporates target hardware feedback directly into the search objective. Rather than optimizing solely for accuracy, the search algorithm queries a latency lookup table or an on-device profiler to estimate inference time, energy consumption, or memory footprint for each candidate architecture. This enables the discovery of Pareto-optimal models that balance classification accuracy against real-time constraints on FPGA fabric. For RF modulation recognition, hardware-aware NAS can discover architectures that maximize throughput on DSP48 slices while meeting strict microsecond-level latency budgets.

The set of all possible neural network architectures that the NAS algorithm can explore. A well-designed search space defines the primitive operations (e.g., depthwise separable convolutions, dilated convolutions, identity skip connections) and their connectivity patterns. For FPGA deployment, the search space often includes hardware-friendly operations such as standard convolutions with small kernel sizes, global average pooling, and ReLU activations that map efficiently to DSP slices and LUT resources. The granularity can range from macro-level (entire blocks) to micro-level (individual layer types and hyperparameters).

A training-efficient approach where a single large supernet encompasses the entire search space as subgraphs. All candidate architectures share weights within this supernet, eliminating the need to train each candidate from scratch. During search, sub-networks are sampled and evaluated using the inherited weights. This reduces the computational cost of NAS from thousands of GPU-hours to a single training run. DARTS (Differentiable Architecture Search) is a prominent one-shot method that relaxes the discrete architecture selection into a continuous optimization problem using softmax-weighted operation mixing.

Optimization algorithms that simultaneously pursue multiple, often conflicting objectives such as classification accuracy, inference latency, and model size. Common strategies include:

• Reinforcement Learning (RL): A controller RNN generates architecture strings and receives a reward signal combining accuracy and hardware efficiency metrics.
• Evolutionary Algorithms: Population-based methods that mutate and crossover architectures, selecting for Pareto-dominant individuals using non-dominated sorting.
• Bayesian Optimization: Builds a probabilistic surrogate model of the objective landscape to efficiently guide the search toward promising regions with minimal evaluations.

A gradient-based NAS method that transforms the discrete architecture search problem into a continuous optimization task. Each edge in a directed acyclic graph (DAG) is assigned a mixture of candidate operations weighted by architectural parameters. Both the network weights and the architectural parameters are optimized simultaneously using bi-level optimization. After convergence, the strongest operation per edge is selected via argmax. DARTS dramatically accelerates search but can suffer from skip-connection collapse, where the algorithm greedily favors parameter-free operations, degrading final performance.

Techniques for rapidly evaluating candidate architectures without full training, critical for making NAS computationally tractable:

• Proxy Tasks: Train on a reduced dataset, for fewer epochs, or at lower resolution to approximate final ranking.
• Zero-Cost Proxies: Compute metrics like Jacobian covariance or NASWOT (Neural Architecture Search Without Training) from a single minibatch at initialization to predict final accuracy.
• Learning Curve Extrapolation: Train partially and fit parametric curves to forecast converged performance, enabling early stopping of unpromising candidates.
• Hardware-in-the-Loop Profiling: Deploy candidate graphs to the target FPGA for real latency measurement, feeding results back into the search loop.