Compute Constraint

Measured in Floating-Point Operations Per Second (FLOPS) or Operations Per Second (OPS), this is the raw computational throughput of a device's CPU, GPU, or NPU. For TinyML devices like MCUs, this is often in the kilo-FLOP (kFLOPS) to mega-FLOP (MFLOPS) range, severely limiting:

• The complexity of neural network layers (e.g., large dense or convolutional layers).
• The feasibility of on-device training versus just inference.
• The latency for completing a local training round within an availability window.

The memory footprint encompasses all volatile (RAM) and non-volatile (Flash) memory consumed. TinyML devices often have < 1 MB of total memory, creating a strict hierarchy of constraints:

• Model Size: Parameters must fit in limited Flash storage.
• Activation Memory: Intermediate layer outputs during forward/backward pass must fit in RAM.
• Runtime Buffers: Space for optimizers (e.g., SGD momentum), gradients, and temporary data. Techniques like model sparsification and quantization are essential to reduce this footprint.

The energy budget is the total joules allocated for a compute task, directly dictating device operational lifetime. Power consumption is non-linear and varies by operation:

• Dynamic Power: Scales with clock frequency and voltage (~CV²f), dominant during active compute.
• Static Power: Constant leakage current, significant during idle.
• Communication Power: Often exceeds computation power; a single radio transmission can consume more than millions of arithmetic operations. Algorithms must minimize energy-per-update to prevent excessive battery drain.

Closely tied to power, thermal constraints limit sustained performance. Without active cooling, embedded devices risk:

• Thermal Throttling: The processor automatically reduces clock speed to lower temperature, drastically increasing computation time.
• Hardware Degradation: Sustained overheating reduces silicon lifespan.
• Unreliable Computation: Excessive heat can cause bit errors in memory and logic. Federated learning protocols must schedule intensive training to avoid prolonged peak thermal loads.

A federation consists of heterogeneous clients with vastly different compute profiles. This variation necessitates:

• Adaptive Algorithms: Methods that account for different client capabilities (e.g., assigning smaller models to weaker devices).
• Asynchronous Aggregation: To mitigate the straggler problem caused by slower devices.
• Portable Model Formats: Representations like TensorFlow Lite for Microcontrollers that can be compiled across different MCU architectures (ARM Cortex-M, RISC-V).

A core design principle in federated edge learning is optimizing the balance between local computation and network communication. Strategies include:

• Increasing Local Epochs: More computation per round reduces total communication rounds needed for convergence.
• Sparse Updates: Transmitting only a fraction of non-zero gradients reduces communication volume.
• Computation-Aware Client Selection: Prioritizing devices with sufficient compute to complete work within a time window. The optimal point minimizes total time-to-accuracy while respecting each device's compute constraint.

Quantization reduces the numerical precision of a model's weights and activations, drastically cutting memory footprint and accelerating computation. This is essential for on-device training.

• Post-Training Quantization (PTQ): Converts a pre-trained model to low-precision (e.g., 8-bit integer) format using a small calibration dataset. Fast but can cause accuracy loss.
• Quantization-Aware Training (QAT): Simulates quantization during training, allowing the model to learn robust, lower-precision representations. Achieves higher accuracy than PTQ but requires a full training cycle.
• Integer-Only Inference/Training: Uses pure integer arithmetic, eliminating power-hungry floating-point units (FPUs) common on MCUs, enabling execution on the broadest hardware.

These techniques remove redundant parameters from a neural network, creating a sparse model that requires fewer computations and less memory.

• Weight Pruning: Identifies and removes less important weights (parameters). Unstructured pruning sets individual weights to zero; structured pruning removes entire neurons or channels for more reliable hardware acceleration.
• Model Sparsification: Induces sparsity during or after training. A sparse model's storage can be compressed, and specialized kernels can skip zero-value computations.
• Sparse Updates in FL: In federated learning, clients can transmit only the non-zero gradient changes (sparse updates), slashing communication costs—a dual benefit for compute and bandwidth constraints.

Designing models specifically for constrained hardware from the ground up is more effective than compressing large models. Key architectures include:

• MobileNet & EfficientNet: Use depthwise separable convolutions to reduce parameters and FLOPs (floating-point operations) while maintaining accuracy for vision tasks.
• Transformer Optimizations: For Small Language Models (SLMs), techniques like grouped-query attention and sliding window attention reduce the quadratic complexity of self-attention.
• Micro-Architectures: Designing ultra-tiny models (e.g., < 50KB) that fit within MCU memory budgets for specific sensor-based tasks (keyword spotting, anomaly detection).

Standard training algorithms like SGD are inefficient on edge devices. Federated optimization introduces client-side efficiency.

• Federated Averaging (FedAvg): The foundational algorithm. Clients perform multiple local epochs of training on their data before sending an update, amortizing the communication cost over more computation. This is tolerant of partial participation and heterogeneous clients.
• Adaptive Client-Side Optimization: Using memory-efficient optimizers like low-precision Adam or sign-based SGD that are robust to the noise and non-IID data characteristics of federated edge learning.
• Gradient Compression: Techniques like top-k sparsification or ternary compression are applied to the local gradients before they are sent to the server, reducing the communication payload.

The full software pipeline must be optimized for the target microcontroller's capabilities.

• TinyML Stack: Frameworks like TensorFlow Lite for Microcontrollers provide optimized kernels for common MCU cores (Arm Cortex-M).
• Embedded FL Runtime: A lightweight library on the device that manages the federated client protocol, model updates, and secure communication within tight memory limits.
• Hardware-Specific Compilation: Using compilers like Apache TVM or proprietary vendor tools to compile models into highly efficient machine code that leverages available hardware accelerators (e.g., Arm Ethos-U55 NPU) or DSP extensions.

Intelligently managing when and how compute-intensive tasks run to respect device constraints and availability.

• Availability Window Training: Scheduling local training rounds only when the device is idle, plugged in, or has excess computational headroom to avoid disrupting primary functions.
• Dynamic Voltage and Frequency Scaling (DVFS): Adjusting the processor's clock speed and voltage in real-time to trade off training speed for power consumption, managing the energy budget.
• Thermal-Aware Throttling: Proactively reducing computational load to prevent thermal throttling, which can cause unpredictable training delays and potential hardware damage on sealed embedded devices.

An embedded system, such as a microcontroller-based sensor, with severe limitations in available memory, processing power, energy supply, and network bandwidth. This defines the primary challenge space for TinyML and federated edge learning. Examples include:

• Microcontroller Units (MCUs) like the Arm Cortex-M series.
• Battery-powered IoT sensors with duty-cycled operation.
• Wearable devices with strict thermal and size limits.

The total amount of volatile (RAM) and non-volatile (Flash) memory consumed by a machine learning model's parameters, activations, and runtime buffers. For TinyML, this is a critical constraint that dictates model feasibility.

• Model Parameters: Stored in Flash memory.
• Runtime Activations: Stored in RAM during inference/training.
• TinyML targets often have < 1MB of total memory, requiring aggressive compression.

The total amount of electrical energy allocated for a computational task, such as model training or inference. A fundamental design constraint for battery-powered devices that directly impacts operational lifetime.

• Battery Drain from compute and radio use is a key metric.
• Thermal Throttling can occur if compute generates excessive heat.
• Algorithms must maximize computations per joule.

Performing neural network computations using numerical formats with fewer bits than standard 32-bit floating-point to reduce memory, power, and latency. Essential for on-device execution.

• 8-bit integer (INT8) and 16-bit float (FP16) are common targets.
• Enabled by Quantization-Aware Training (QAT) and Post-Training Quantization (PTQ).
• Integer-Only Inference eliminates need for floating-point hardware.

The process of inducing sparsity in a neural network by zeroing out a significant fraction of its parameters (weights or gradients). Creates a model that can be stored and computed more efficiently.

• Weight Pruning removes unimportant parameters.
• Enables Sparse Updates in federated learning, reducing communication cost.
• Requires specialized libraries or hardware to exploit sparsity.

The variation in hardware capabilities (compute, memory), data distributions, network connectivity, and availability among participating edge devices in a federation. This complicates synchronous training.

• Leads to the Straggler Problem, where slow clients delay aggregation.
• Requires algorithms that support Partial Participation.
• Must account for different Availability Windows for training.