Short-Time Fourier Transform (STFT)

The window function (e.g., Hann, Hamming, Blackman) is multiplied with each segment of the signal before the FFT to reduce spectral leakage, a distortion where energy from one frequency spreads into neighboring bins. The choice of window involves a direct trade-off:

• Main lobe width: Determines frequency resolution; wider lobes smear nearby frequencies together.
• Side lobe attenuation: Determines how well the window suppresses leakage from distant frequencies.
• TinyML Implication: Simpler windows (like Hann) offer a good balance and are computationally cheaper to apply on a microcontroller than more complex ones.

The window length (N) is the number of samples in each analysis segment. It directly defines the frequency resolution (Δf) of the resulting spectrogram: Δf = Fs / N, where Fs is the sampling rate.

• Longer Windows: Provide finer frequency resolution (smaller Δf), better for distinguishing close tones.
• Shorter Windows: Provide coarser frequency resolution, but better time localization for transient events.
• TinyML Constraint: Longer windows require larger FFTs, increasing memory usage and compute time quadratically (O(N log N)). On an MCU, N is often severely limited by RAM.

The hop size (H) is the number of samples the window advances between successive STFT frames. It determines the temporal resolution (or overlap) of the spectrogram.

• Smaller Hop (High Overlap): Produces a smoother, more detailed spectrogram in time but increases the total number of frames to compute.
• Larger Hop (Low/No Overlap): Reduces computational load and output data rate but can cause temporal aliasing, missing short-duration events.
• TinyML Optimization: A hop size of N/2 (50% overlap) is common, but for ultra-low-power applications, H = N (no overlap) may be necessary to minimize compute.

The Heisenberg-Gabor limit (or uncertainty principle) imposes a fundamental constraint: one cannot simultaneously achieve arbitrarily high resolution in both time and frequency. This is the core trade-off governing STFT parameter selection.

• Fine Frequency Resolution requires a long time window, which blurs the timing of rapid changes.
• Fine Time Resolution requires a short window, which spreads out frequency information.
• Engineering Implication: Parameters must be chosen based on the signal's characteristics. Analyzing a steady hum requires high frequency resolution; detecting a brief click requires high time resolution.

The FFT size (M) is often larger than the window length (N). The difference is filled with zero-padding, which appends zeros to the windowed segment before the FFT.

• Purpose: Zero-padding interpolates the frequency spectrum, providing a smoother visual representation and allowing frequency bins to be centered on specific frequencies of interest.
• Critical Note: It does not improve true frequency resolution (which is fixed by N). It only provides interpolated bins.
• TinyML Cost: Increasing M increases FFT computation time. Efficient libraries use M as a power of two. The choice is a balance between spectral smoothness and MCU cycle count.

The STFT's computational load on a microcontroller is dominated by the window multiply and the FFT for each frame.

• Operations per Frame: ~N (window multiply) + O(M log₂ M) (FFT).
• Frames per Second: Fs / H.
• Total Load: Scales with (Fs / H) * (N + M log₂ M).
• Optimization Strategies for TinyML:
  • Use fixed-point arithmetic instead of floating-point.
  • Pre-compute and store the window function in ROM.
  • Use a highly optimized, power-of-two FFT library (e.g., CMSIS-DSP for ARM Cortex-M).
  • Carefully select N, M, and H to stay within real-time deadlines and memory limits.

The Fast Fourier Transform (FFT) is an efficient algorithm for computing the Discrete Fourier Transform (DFT), which decomposes a signal from its time domain into its constituent frequencies. It is the computational engine behind each windowed segment in the STFT.

• Core Relationship: The STFT applies the FFT repeatedly to overlapping, windowed segments of a signal.
• Efficiency: The FFT's O(N log N) complexity makes real-time STFT feasible on microcontrollers.
• Constraint: The standard FFT assumes stationarity, which the STFT overcomes by analyzing local, short-time windows.

A spectrogram is the visual or data representation produced by the STFT, plotting signal frequency content against time. It is the primary output used for analysis and feature extraction.

• Axes: Time (x-axis), Frequency (y-axis), and Intensity/Energy (color or z-axis).
• Applications: Used for visualizing audio patterns, vibration analysis, and as input features for convolutional neural networks (CNNs) in TinyML.
• Optimization: On-device, spectrograms are often computed using fixed-point arithmetic and stored in quantized formats to minimize memory footprint.

A windowing function (e.g., Hann, Hamming, Blackman) is applied to each segment of data before the FFT in the STFT process to minimize spectral leakage.

• Purpose: Tapers the edges of the signal segment to reduce artifacts caused by analyzing a non-integer number of cycles.
• Trade-off: Introduces a resolution trade-off between frequency precision and time localization.
• TinyML Impact: Window choice and pre-computed window tables affect accuracy and compute load on microcontrollers.

The Wavelet Transform is an alternative time-frequency analysis method that uses variable-sized windows, providing multi-resolution analysis.

• Key Difference: Unlike the STFT's fixed window, wavelets use short windows at high frequencies and long windows at low frequencies.
• Advantage: Better suited for analyzing signals with sudden, transient events (e.g., impacts, clicks).
• Computational Consideration: Can be more computationally intensive than STFT, making STFT often preferred for predictable, real-time streams on MCUs.

Mel-Frequency Cepstral Coefficients (MFCCs) are a compact feature set derived from an STFT-based spectrogram, designed to mimic human auditory perception.

• Processing Pipeline: STFT → Mel-filter bank (warping to Mel scale) → Logarithm → Discrete Cosine Transform (DCT).
• Primary Use: The standard feature for speech and audio recognition models, including those deployed on edge devices.
• Efficiency: MFCCs drastically reduce the dimensionality of the raw spectrogram, making them ideal for TinyML models with limited input size.

The time-frequency resolution trade-off is a fundamental principle (related to the Heisenberg-Gabor limit) governing all time-frequency analyses like the STFT.

• Window Length: A long window provides good frequency resolution but poor time localization.
• Short Window: Provides good time localization but poor frequency resolution.
• Design Implication: Selecting the STFT window size and overlap is a critical system design choice for sensor applications, balancing the need to detect rapid events versus precisely identifying their frequency components.