CMSIS-DSP

The library offers a wide range of signal processing functions essential for feature extraction in TinyML. Key categories include:

• Basic Math Functions: Vector addition, multiplication, dot products.
• Fast Math Functions: Optimized sine, cosine, square root.
• Complex Math Functions: Operations on complex number vectors.
• Filters: Finite Impulse Response (FIR), Infinite Impulse Response (IIR), Biquad, and lattice filters.
• Matrix Functions: Addition, multiplication, transposition for small matrices common in ML.
• Transform Functions: Fast Fourier Transform (FFT) for frequency analysis (Q15, Q31, floating-point).
• Statistical Functions: Mean, variance, standard deviation, RMS.
• Support Functions: Data type conversions (e.g., float to Q15) and copy operations.

A core feature for microcontroller deployment is extensive support for fixed-point arithmetic, which avoids the computational cost of floating-point units. CMSIS-DSP uses Q-format representations (Q7, Q15, Q31) where numbers are treated as integers with an implicit binary point. Functions are provided for:

• Q7, Q15, and Q31 data types for different precision/range trade-offs.
• Saturation arithmetic to prevent overflow.
• Conversion functions between floating-point and fixed-point formats.
• This is critical for implementing efficient neural network operations and sensor data preprocessing on cores lacking an FPU.

All CMSIS-DSP functions are designed for deterministic execution, with no dynamic memory allocation and predictable, bounded execution times. This is a non-negotiable requirement for real-time embedded systems and TinyML applications processing continuous sensor streams. The library's static memory footprint allows it to be integrated into safety-critical systems certified under standards like IEC 61508 or ISO 26262. Functions operate on arrays provided by the caller, ensuring no hidden heap usage.

CMSIS-DSP serves as the computational foundation for CMSIS-NN, Arm's optimized neural network kernel library. Many TinyML preprocessing stages rely on CMSIS-DSP functions:

• FFT for audio keyword spotting or vibration analysis.
• FIR/IIR filters for noise reduction on sensor signals.
• Matrix operations for implementing fully connected layers.
• Statistical functions for sensor data normalization. Using a unified, optimized library for both DSP and NN operations reduces code size and leverages the same highly tuned low-level arithmetic kernels across the entire signal chain.

CMSIS-DSP is fundamental for extracting spectral features from audio signals, which are the inputs for keyword spotting or audio event detection models. Key functions include:

• Fast Fourier Transform (FFT): Converts time-domain audio samples to the frequency domain.
• Mel-Frequency Cepstral Coefficients (MFCC) computation: Uses FFT, Mel filter banks, and Discrete Cosine Transform (DCT) to produce compact, perceptually relevant features.
• Windowing and filtering: Applies Hann or Hamming windows and band-pass filters to condition the signal before analysis. This preprocessing reduces the complexity the neural network must learn, enabling smaller, more efficient models.

For predictive maintenance and activity recognition using accelerometers and gyroscopes, CMSIS-DSP processes inertial measurement unit (IMU) data in real-time. Common operations include:

• Digital filtering: Low-pass, high-pass, and band-pass filters (using FIR/IIR functions) remove sensor noise and isolate frequency bands of interest.
• Root Mean Square (RMS) & statistical feature calculation: Computes time-domain features like mean, variance, and peak-to-peak amplitude over a sliding window.
• Frequency-domain analysis: Uses FFT to detect specific resonant frequencies indicative of motor faults or equipment wear. These processed features are fed into classifiers to identify anomalies or specific human activities.

For microcontroller-based computer vision, CMSIS-DSP performs low-level pixel operations before feeding data into a vision model. This is critical for resource-constrained systems. Functions include:

• Color space conversion: Efficient transformation from RGB to grayscale or other color models using fixed-point arithmetic.
• Image filtering and convolution: Applies kernels for edge detection (Sobel), blurring, or sharpening directly on the sensor output.
• Image resizing and cropping: Uses bilinear interpolation functions to downsample images to the model's required input dimensions.
• Pixel normalization: Scales pixel values to a fixed-point range suitable for the quantized neural network input. This preprocessing offloads work from the neural network, allowing for a simpler model architecture.

In applications combining multiple sensors (e.g., IMU, magnetometer, pressure), CMSIS-DSP implements sensor fusion algorithms to create a stable, unified state estimate. Key algorithms include:

• Basic Linear Algebra Subprograms (BLAS): Matrix and vector operations (addition, multiplication) essential for Kalman filters.
• Quaternion and rotation matrix math: Functions for efficiently combining orientation data from different sensors.
• Interpolation and decimation: Aligns data streams from sensors sampling at different rates. This fused, clean data provides a more robust and accurate input for downstream TinyML decision models.

In closed-loop embedded systems with a TinyML-based controller, CMSIS-DSP conditions the feedback signal. This ensures stable and responsive control. Typical uses are:

• PID controller implementation: Provides the proportional, integral, and derivative calculations using fixed-point arithmetic for deterministic timing.
• Noise reduction: Applies real-time digital filters to sensor feedback before it is evaluated by the ML model or control logic.
• Signal smoothing: Uses moving average or median filters to eliminate transient spikes that could cause erratic control outputs. This role highlights CMSIS-DSP as the bridge between the physical sensor signal and the intelligent control algorithm.

Before transmitting sensor data or storing it locally, CMSIS-DSP can compress it to save energy and bandwidth. Techniques include:

• Lossless compression: Implementations of algorithms like run-length encoding (RLE) on sensor data sequences.
• Principal Component Analysis (PCA): Uses matrix decomposition functions (SVD) to reduce the dimensionality of feature vectors while preserving most of the signal information.
• Delta encoding: Computes the difference between consecutive samples, which often has lower entropy and compresses better. This enables more efficient data logging or communication in battery-powered IoT nodes.

Fixed-point arithmetic is a method for representing fractional numbers using integers, crucial for predictable and efficient computation on microcontrollers lacking Floating-Point Units (FPUs).

• CMSIS-DSP provides extensive support for Q-format fixed-point operations (e.g., Q15, Q31).
• Eliminates the overhead and power consumption of software-emulated floating-point math.
• Essential for maintaining performance in real-time digital signal processing pipelines for audio, vibration, and other sensor data.

Digital Signal Processing (DSP) is the computational manipulation of discrete-time signals to filter, analyze, or transform information. It is the foundational discipline behind CMSIS-DSP.

• Core operations include Fast Fourier Transform (FFT), Finite Impulse Response (FIR) filters, and matrix functions.
• In TinyML, DSP is used for feature extraction (e.g., computing Mel-Frequency Cepstral Coefficients for audio) to reduce raw sensor data into meaningful inputs for a neural network.
• CMSIS-DSP provides the optimized building blocks for these real-time preprocessing stages.