Memory Footprint

The static memory required to store the learned parameters of the neural network. This is the model's persistent size on non-volatile storage (e.g., Flash).

• Primary Driver: Number of parameters and their numerical precision (e.g., FP32, INT8).
• TinyML Impact: Often the largest single component. Reduced via quantization, pruning, and efficient architectures (e.g., depthwise separable convolutions).
• Example: A 250KB Flash budget might allow for ~65,000 INT8 parameters, dictating model architecture choices.

The peak runtime memory (RAM) required to store the outputs of each layer during a single forward or backward pass. This is often the limiting factor for inference and training on MCUs.

• Defined by Model Architecture: Sequence length, number of channels, and feature map dimensions.
• Critical for Training: Backpropagation can require saving activations for the entire computation graph, significantly increasing peak RAM vs. inference-only.
• Optimization Techniques: Activation quantization, gradient checkpointing (re-computing activations), and operator fusion to eliminate intermediate buffers.

Memory allocated by the inference/training engine for temporary computations, kernel execution, and I/O buffering. This is overhead beyond the model itself.

• Includes: Scratch space for convolution kernels, matrix multiplication intermediates, and sensor data input buffers.
• Framework-Dependent: Different TinyML runtimes (TensorFlow Lite Micro, CMSIS-NN) have varying overhead.
• Embedded FL Runtime: Adds buffers for the local optimizer state (e.g., momentum), received global model, and computed update.

Memory required for the local training process in Federated Edge Learning. This expands the footprint far beyond inference.

• Key Components:
  • Local Optimizer State: Gradients, momentum, and variance accumulators (e.g., for SGD, Adam). Can be 2x-3x the model parameter size.
  • Local Dataset Buffer: Storage for batches of sensor data used in each local epoch.
  • Backward Pass Graph: Storing computation graphs for gradient calculation.
• Consequence: A device capable of inference may lack the RAM for training, forcing algorithmic adaptations like federated fine-tuning of only final layers.

Memory reserved for sending and receiving model updates in a federated learning system. Must be managed within the same constrained RAM.

• Update Buffer: Holds the computed model update (e.g., weight delta, gradients) before encryption and transmission.
• Receive Buffer: Stores the incoming global model or aggregated update from the server.
• Optimization Driver: Techniques like sparse updates (sending only top-k gradients) or structured updates are primarily designed to shrink this buffer, reducing both communication cost and peak RAM usage.

The baseline memory consumed by the device's operating system, communication stacks, and application firmware, which must be subtracted from the total available resources.

• Fixed Costs: RTOS kernel, BLE/Wi-Fi stack, sensor drivers, and security protocols (e.g., TLS, secure aggregation).
• TinyML Implication: A device datasheet listing 256KB of SRAM may only have 150KB available for the ML application after system overhead.
• Integration Challenge: The ML model and runtime must be compiled and linked into the monolithic firmware image, requiring careful memory mapping and section placement.

A compute constraint is the limitation imposed by the available processing power (e.g., operations per second) of a microcontroller, which restricts the complexity of models and training algorithms that can be executed locally. For federated edge learning, this directly caps:

• The size and architecture of trainable neural networks.
• The number of local training epochs per federated round.
• The feasibility of advanced on-device optimization techniques. Exceeding this constraint leads to intractable training times or thermal throttling.

The energy budget is the total electrical energy allocated for a computational task, a fundamental design limit for battery-powered devices. In federated TinyML, the most energy-intensive operations are:

• Radio communication for transmitting model updates.
• On-device training cycles.
• Continuous sensor sampling. Engineers must trade off model complexity, training frequency, and communication volume against the target device lifespan, often measured in months or years.

Post-Training Quantization (PTQ) is a compression technique that converts a pre-trained model's weights and activations from 32-bit floating-point to lower-precision formats (e.g., 8-bit integers) after training. For deployment, PTQ:

• Reduces memory footprint by 4x for 8-bit vs. 32-bit.
• Enables integer-only inference on hardware without floating-point units.
• Requires a small calibration dataset to determine optimal scaling factors. It is a critical, often final, step for fitting models into MCU flash memory.

Weight pruning is a model compression technique that removes less important parameters from a neural network to reduce its size and computational cost. For TinyML, two primary methods are used:

• Unstructured Pruning: Sets individual weights to zero, creating sparsity. Requires specialized sparse compute libraries for real speedup.
• Structured Pruning: Removes entire neurons, channels, or layers, leading to directly smaller, dense models that are easier to deploy. Pruning is often combined with quantization for maximum footprint reduction.

An embedded Federated Learning (FL) runtime is a lightweight software library that runs on a microcontroller to manage the local client protocol. Its memory footprint is critical and includes:

• The model parameters in RAM/Flash.
• Runtime buffers for activations and gradients.
• Code for the optimizer (e.g., SGD), loss function, and secure communication handlers. This runtime must be integrated into the device's firmware, competing for space with the core application and sensor drivers.

A resource-constrained device is an embedded system, typically a microcontroller unit (MCU), with severe limitations that define the TinyML problem space:

• Memory: Often < 1 MB of Flash (for model storage) and < 512 KB of RAM (for runtime execution).
• Compute: Clock speeds in the low hundreds of MHz, without GPUs.
• Power: Milliwatt-scale operation, often battery-powered.
• Connectivity: Intermittent, low-bandwidth links (e.g., LoRaWAN, BLE). Federated learning algorithms must be co-designed with these absolute hardware limits.