Memory Constraint (NAS)

This is the maximum RAM required to store the intermediate feature maps (activations) during a single forward pass of inference. It is often the dominant memory constraint for convolutional networks on microcontrollers.

• Determined by: Layer output dimensions, batch size (typically 1 for MCUs), and data type precision (e.g., int8 vs. float32).
• Critical for: Real-time inference, as exceeding available RAM causes system crashes. Search algorithms must optimize layer widths and spatial resolutions to control this peak.

This refers to the non-volatile storage (e.g., flash memory) required for the static model weights and biases. It is a fixed cost determined by the final architecture.

• Calculated as: (Number of parameters) * (Size per parameter in bytes). For example, 100k int8 parameters consume ~100KB.
• NAS Impact: The search must favor efficient operations like depthwise separable convolutions and prune unnecessary channels to minimize parameter count while preserving accuracy.

NAS integrates memory as a primary optimization target, often within a multi-objective framework. The search goal is not just accuracy, but architectures that respect hard limits.

• Common Formulations:
  • Hard Constraint: Accuracy, subject to PeakRAM < 64KB.
  • Soft Penalty: Reward = Accuracy - λ * MemoryOverhead.
• Pareto Frontier: The search outputs a set of models representing the best accuracy-memory trade-offs, allowing engineers to select the optimal fit for their hardware.

To evaluate memory usage without deploying every candidate, NAS relies on predictive hardware cost models. These are lightweight estimators that map an architecture description to its predicted memory footprint.

• Types of Models:
  • Analytical Models: Rule-based calculators that sum memory from known layer specifications.
  • Learned Models: Small neural networks trained on profiled data from the target hardware.
• Purpose: Drastically accelerates the search by providing fast, approximate memory evaluations, guiding the algorithm toward feasible regions of the search space.

The set of possible architectures (the search space) is explicitly designed to be memory-efficient. Constraining the space a priori makes the search more tractable.

• Key Design Levers:
  • Operator Choice: Prioritizing memory-efficient ops (e.g., depthwise conv over standard conv).
  • Channel Search: Allowing the algorithm to choose layer widths, directly controlling activation and parameter size.
  • Kernel Size Search: Smaller kernels (3x3 vs. 5x5) reduce parameter count.
• Pruning: Invalid architectures that exceed pre-calculated memory budgets are pruned from the search space early.

NAS for microcontrollers (MCU-NAS) deals with extreme, kilobyte-scale budgets that introduce unique challenges absent in mobile or cloud NAS.

• Tight Coupling: Memory for activations, parameters, and runtime stack often shares a single, small SRAM bank.
• Static Allocation: Many MCU runtimes require static memory allocation, meaning the peak activation memory must be reserved at compile time and cannot be dynamically managed. This makes precise prediction critical.
• Quantization-Aware Search: Memory constraints make low-precision arithmetic (int8) essential. Quantization-Aware NAS (QA-NAS) searches for architectures that are robust to the accuracy loss from quantization, which halves or quarters memory footprints.

A predictive function used within NAS to estimate the latency, energy, or memory usage of a candidate architecture without deploying it on physical hardware. For memory constraints, this model predicts peak RAM (activation) and flash (model weight) usage.

• Implementation: Often a lookup table (LUT) populated from profiling basic operations or a small neural network regressor.
• Purpose: Drastically accelerates search by replacing slow, physical measurements with instant estimates, guiding the algorithm toward memory-feasible designs.

An automated design process that optimizes a neural network across multiple, often competing objectives. Memory usage is a primary objective alongside accuracy, latency, and energy consumption.

• Trade-off Management: The search finds Pareto-optimal architectures where improving one metric (e.g., accuracy) worsens another (e.g., memory footprint).
• Constraint Formulation: A memory constraint can be a hard limit (strict budget) or a soft objective weighted against others in a compound loss function.

The set of all possible neural network architectures the NAS algorithm can explore. For memory-constrained search, this space is explicitly designed to include only microcontroller-feasible operations.

• Key Parameters: Includes choices for layer type (depthwise conv vs. standard conv), kernel size, number of channels, and skip connections.
• Pruning: The search space is often pruned to exclude operations known to exceed memory budgets (e.g., large dense layers), making the search more efficient.

A hardware-aware search method that discovers architectures robust to post-training quantization. Since quantization reduces model weight size (flash memory), QA-NAS co-optimizes for accuracy under quantized weights.

• Process: The search trains or evaluates candidates using simulated quantization (e.g., 8-bit integers).
• Outcome: Finds architectures that maintain high accuracy after quantization, ensuring the discovered model's final deployed size fits the memory constraint.

A one-shot NAS and training paradigm that produces a single, large supernet from which many efficient, hardware-tailored sub-networks can be extracted without retraining.

• Memory Constraint Application: After training the supernet, a specialized search (e.g., using a hardware cost model) selects a sub-network that meets a specific kilobyte-scale memory budget.
• Efficiency: Enables rapid deployment of different models for various MCU memory tiers from one trained network.

Hardware-aware neural architecture search methods specifically tailored to the extreme constraints of microcontrollers. It directly optimizes for kilobyte-scale RAM, flash memory, and microjoule-level energy.

• Distinct Challenges: Focuses on extreme model compression, fixed-point arithmetic efficiency, and minimal runtime memory for activations.
• Typical Output: Produces deeply quantized, pruned architectures using operators like depthwise separable convolutions that are standard in MCU-optimized networks.