Wavelet Transform

The Wavelet Transform's defining feature is its ability to analyze a signal at multiple scales or resolutions. Unlike the Fourier Transform, which provides a single frequency resolution for the entire signal, wavelet analysis uses a variable-sized window:

• High frequencies (fine details) are analyzed with a narrow wavelet, providing good time localization.
• Low frequencies (broad trends) are analyzed with a wide wavelet, providing good frequency localization. This hierarchical decomposition is fundamental for applications like image compression (JPEG 2000) and detecting transient events in vibration data.

Wavelets are localized in both time and frequency. Each wavelet basis function is a brief oscillation that decays rapidly to zero. This contrasts with the infinite sine waves of the Fourier Transform.

• Heisenberg Uncertainty Principle: There is a trade-off between time and frequency precision; wavelets offer a flexible compromise.
• Practical Implication: A wavelet coefficient tells you when (time) a signal component of a certain scale (inverse of frequency) occurs. This is critical for pinpointing the onset of a fault in machinery or the exact moment a keyword is spoken.

The transform has two primary computational variants:

• Continuous Wavelet Transform (CWT): Uses a continuous range of scales and translations. It produces a highly redundant representation useful for detailed time-frequency analysis and feature detection.
• Discrete Wavelet Transform (DWT): Uses a discrete set of scales and translations, typically dyadic (powers of two). It is non-redundant, computationally efficient, and forms the basis for multiresolution analysis and practical implementations like filter banks. The DWT's efficiency makes it suitable for real-time processing on resource-constrained devices.

A wavelet's number of vanishing moments is a key mathematical property. A wavelet has p vanishing moments if it is orthogonal to polynomials of degree up to p-1.

• Higher vanishing moments mean the wavelet can better represent smooth parts of a signal, leading to many near-zero coefficients and efficient compression.
• Lower vanishing moments preserve sharp transitions and discontinuities. Engineers select wavelets (e.g., Daubechies, Haar) based on this property to match the signal characteristics, balancing compression efficiency against artifact creation.

For the Discrete Wavelet Transform, the analysis filter bank (which decomposes the signal) and the synthesis filter bank (which reconstructs it) are designed for perfect reconstruction. This means the original signal can be exactly reconstructed from its wavelet coefficients, barring numerical errors.

• This property is non-negotiable for lossless compression and critical signal analysis.
• It relies on the careful design of quadrature mirror filters (QMF) that satisfy specific mathematical conditions. This ensures transformations are invertible and information-preserving.

For many natural signals (images, audio, sensor readings), the Wavelet Transform provides a sparse representation. This means most of the signal's energy is concentrated in a small number of large-magnitude coefficients, while the majority are near zero.

• Sparsity enables compression: Small coefficients can be quantized or set to zero with minimal perceptual impact (e.g., in the JPEG 2000 image standard).
• Efficient Storage/Transmission: Only significant coefficients need to be stored or transmitted.
• Denoising: Techniques like wavelet thresholding exploit sparsity to separate signal from noise by zeroing out small coefficients assumed to be noise.

Wavelet coefficients serve as highly informative features for classifying human activities from Inertial Measurement Unit (IMU) data. Unlike the Fast Fourier Transform (FFT), which assumes signal stationarity, wavelets capture transient events like steps or jumps.

• Key Advantage: Provides multi-resolution analysis, isolating both slow movements (e.g., walking) and fast, impulsive motions (e.g., a jump) within the same transform.
• TinyML Benefit: The transform can be computed efficiently on streaming data using a sliding window, and only the most discriminative coefficients need to be stored for model inference, minimizing memory footprint.
• Example: Decomposing accelerometer data to distinguish between 'walking', 'running', and 'falling' with high temporal precision.

Wavelet-based compression is critical for transmitting or storing high-sample-rate sensor data from condition monitoring systems. It achieves high compression ratios while preserving fault signatures.

• Mechanism: The transform concentrates signal energy into a few large coefficients. Thresholding discards small coefficients (often noise), and the remaining significant coefficients are encoded.
• TinyML Application: Enables Anomaly Detection models to run on-premise by reducing the data bandwidth needed to send raw vibration telemetry to a cloud gateway.
• Real-World Use: Monitoring industrial machinery (pumps, motors) for early signs of bearing failure using a MEMS accelerometer, where only compressed wavelet features are transmitted upon detecting an exceedance.

Wavelet threshold denoising is a powerful, computationally manageable method for cleaning noisy signals from low-power sensors before feature extraction or model inference.

• Process: The signal is transformed, wavelet coefficients below a statistically derived threshold are zeroed out (assuming they represent noise), and the signal is reconstructed.
• Advantage over FIR/IIR Filters: Excels at removing non-stationary noise and preserving sharp signal transitions (e.g., edges in a signal, impact spikes) that linear filters would smooth out.
• TinyML Implementation: A single, optimized denoising function can be applied to multiple sensor types (audio, vibration, strain) improving Signal-to-Noise Ratio (SNR) for downstream tasks like Acoustic Event Detection (AED) or Voice Activity Detection (VAD).

In microcontroller-based computer vision, 2D wavelet transforms (like Haar wavelets) provide an efficient method for multi-scale edge detection and texture analysis without the computational burden of full convolutional neural networks.

• Efficiency: The Haar wavelet transform uses only addition and subtraction, making it ideal for fixed-point arithmetic on MCUs. It's a cornerstone of simple image compression and feature extraction.
• Application: Used in Optical Flow calculations or as a pre-processing step for Gesture Recognition systems using low-resolution sensors. It can highlight regions of motion or distinct patterns.
• System Impact: Reduces the input dimensionality for a subsequent tiny neural network, lowering overall inference latency and energy consumption.

Deploying wavelet transforms on microcontrollers requires careful algorithm selection and optimization to meet severe constraints on RAM, flash, and CPU cycles.

• Algorithm Choice: The Discrete Wavelet Transform (DWT) with a short filter (e.g., Haar, Daubechies D4) is preferred for its lower computational complexity versus the Continuous Wavelet Transform (CWT).
• Optimization Techniques:
  • Fixed-Point Arithmetic: Implementing the transform using integer math to avoid slow floating-point units.
  • In-Place Computation: The pyramid algorithm can be designed to operate on a single buffer, minimizing RAM usage.
  • Look-Up Tables (LUTs): Storing filter coefficients in ROM to save computation.
• Library Support: Leveraging optimized DSP libraries in frameworks like CMSIS-DSP for Arm Cortex-M cores.

Understanding when to use Wavelet Transform over Fast Fourier Transform (FFT) or Short-Time Fourier Transform (STFT) is crucial for TinyML system design.

• Time-Frequency Resolution: The FFT provides perfect frequency resolution but no time localization. STFT provides a fixed time-frequency window. Wavelets offer variable resolution—good time resolution at high frequencies, good frequency resolution at low frequencies.
• Use Case Guide:
  • Use FFT/STFT: For stationary signals (e.g., analyzing the constant hum of a motor) or when you need exact frequency bins.
  • Use Wavelets: For signals with transient spikes, discontinuities, or events localized in time (e.g., a bearing fault impact, a keyword in speech, a heart QRS complex).
• TinyML Trade-off: Wavelet analysis can be more computationally intensive than a single FFT but often provides more compact, informative features, leading to a smaller and more accurate classifier.

The Short-Time Fourier Transform (STFT) applies the Fourier Transform to short, overlapping segments of a signal using a fixed-size window. This provides a time-frequency representation called a spectrogram.

• Key Difference from Wavelets: Uses a fixed window size, leading to a uniform time-frequency resolution trade-off across all frequencies.
• Use Case: Ideal for analyzing signals where the frequency content changes relatively slowly over time.
• TinyML Consideration: Computationally intensive due to the FFT on many windows; less efficient for signals with transient, high-frequency events compared to wavelets.

The Fast Fourier Transform (FFT) is an efficient algorithm for computing the Discrete Fourier Transform (DFT), which decomposes a signal from the time domain into its constituent frequencies in the frequency domain.

• Global Analysis: Provides excellent frequency resolution but no time localization—it tells you what frequencies are present in the entire signal, but not when they occur.
• Foundation: The FFT is the computational engine behind both the STFT and many wavelet transform implementations.
• Primary Use: Analyzing stationary signals (where frequency content does not change over time) and for filter design.

Digital Signal Processing (DSP) is the overarching field concerning the algorithmic manipulation of discrete-time signals. It provides the foundational toolbox for sensor data analysis.

• Core Techniques: Includes filtering (FIR, IIR), transforms (FFT, DCT, Wavelet), convolution, correlation, and decimation.
• Relationship to Wavelets: The Wavelet Transform is a specialized DSP tool within this broader discipline, designed for multi-resolution analysis.
• TinyML Criticality: Efficient DSP algorithms are prerequisites for real-time feature extraction on microcontrollers before model inference.

The Discrete Cosine Transform (DCT) expresses a finite sequence of data points as a sum of cosine functions oscillating at different frequencies. It is highly effective for energy compaction.

• Compression Focus: The cornerstone of lossy audio (MP3) and image (JPEG) compression standards.
• Comparison to Wavelets: Like the FFT, the DCT is a global transform. Wavelets often outperform DCT for compressing signals with sharp discontinuities or transients.
• Application: Used in some TinyML pipelines for compressing sensor data or as a feature extraction step for periodic signals.

Multi-Resolution Analysis (MRA) is a framework for analyzing signals at different scales or resolutions. It is the mathematical foundation of the Discrete Wavelet Transform (DWT).

• Core Principle: Represents a signal using a series of approximation coefficients (low-frequency trends) and detail coefficients (high-frequency details) at progressively coarser scales.
• Practical Outcome: Allows a system to examine broad signal trends and fine-grained details separately, which is computationally efficient.
• TinyML Benefit: Enables scalable processing; a device can first analyze a coarse approximation to detect an event, then activate more detailed analysis only if needed, saving power.

Feature Extraction is the process of transforming raw, high-dimensional sensor data into a reduced set of informative, non-redundant values (features) that are optimal for machine learning models.

• Wavelets as Feature Engines: The coefficients from a wavelet decomposition (e.g., energy per sub-band, statistical moments of coefficients) are powerful features for time-series classification, anomaly detection, and activity recognition.
• Dimensionality Reduction: Critical for TinyML, where model input size directly impacts memory and compute requirements. A good wavelet-based feature set can be far more compact than raw data.
• Pipeline Stage: Typically follows signal conditioning (filtering) and precedes model inference in a TinyML workflow.