Digital Signal Processing (DSP)

The Fourier Transform is the fundamental operation that decomposes a signal from its time domain representation into its constituent frequencies in the frequency domain. This reveals which frequencies are present and their relative strengths.

• Core Insight: Any signal can be represented as a sum of simple sine waves.
• Discrete Fourier Transform (DFT): The version used for digital, sampled signals.
• Fast Fourier Transform (FFT): A highly efficient algorithm for computing the DFT, reducing complexity from O(N²) to O(N log N). This efficiency is critical for real-time analysis on microcontrollers.
• Use Case: Identifying a 50 Hz hum in an audio signal or the dominant vibration frequency in a motor.

Digital filters are algorithms designed to remove unwanted components or enhance desired features within a signal. They are categorized by their impulse response and design.

• Finite Impulse Response (FIR) Filters: Output depends only on a finite number of past and present inputs. They are always stable and can be designed to have a linear phase response, which prevents signal distortion.
• Infinite Impulse Response (IIR) Filters: Output depends on past inputs and past outputs. They are more computationally efficient for a given frequency response but require careful design to ensure stability.
• Application: A low-pass FIR filter can remove high-frequency noise from a temperature sensor reading, while a band-pass IIR filter can isolate a specific voice frequency range.

Real-world signals, like audio or vibration, have frequencies that change over time. A single Fourier Transform of the entire signal loses this temporal information.

• Windowing: The technique of applying the FFT to short, consecutive segments of a signal. A window function (e.g., Hamming, Hann) is applied to each segment to minimize spectral leakage at the edges.
• Short-Time Fourier Transform (STFT): The algorithm that performs this windowed FFT across the entire signal.
• Spectrogram: The visual output of the STFT—a 2D plot with time on the x-axis, frequency on the y-axis, and intensity represented by color. This is the primary tool for analyzing non-stationary signals like speech or machinery startup sequences.

The bridge between the analog and digital worlds is defined by sampling. The Nyquist-Shannon sampling theorem is the fundamental law governing this process.

• Sampling Rate (Fs): The frequency at which an ADC measures the continuous analog signal (e.g., 16 kHz for audio).
• Nyquist Frequency: Defined as Fs / 2. It is the highest frequency that can be uniquely represented.
• Aliasing: A destructive artifact that occurs if a signal contains frequency components above the Nyquist frequency before sampling. These high frequencies 'fold back' and appear as lower, false frequencies.
• Anti-Aliasing Filter: A mandatory analog low-pass filter applied before the ADC to remove any frequencies above Nyquist, preventing aliasing.

Before feeding raw sensor data into a machine learning model, DSP is used to extract compact, informative features that reduce dimensionality and improve model efficiency—a critical step for TinyML.

• Time-Domain Features: Calculated directly from the signal samples.
  • Root Mean Square (RMS): Measures signal power.
  • Zero-Crossing Rate (ZCR): Rate of sign changes; high for noise or unvoiced speech.
  • Peak Detection: Identifies local maxima for event detection.
• Frequency-Domain Features: Derived from the FFT.
  • Spectral Centroid: The 'center of mass' of the spectrum.
  • Spectral Roll-off: The frequency below which a specified percentage (e.g., 85%) of the total spectral energy lies.
• Cepstral Features: Mel-Frequency Cepstral Coefficients (MFCCs) are a compact representation that mimics human auditory perception, forming the basis of most speech recognition systems.

For continuous analysis of sensor streams on embedded devices, data is processed incrementally using a sliding window architecture.

• Mechanism: A buffer holds the most recent N samples of data. As each new sample arrives, the oldest sample is discarded, and the window 'slides' forward by one sample.
• Overlap: Windows are often overlapped (e.g., a 256-sample window sliding by 128 samples) to ensure no transient event is missed at the window boundary.
• Processing Loop: For each new window position:
  1. Pre-process (e.g., remove DC offset, apply window function).
  2. Compute Features (e.g., FFT, extract RMS, calculate MFCCs).
  3. Run Inference with the TinyML model on the feature vector.
• Key Constraint: The entire loop must execute faster than the time it takes to collect N new samples to maintain real-time operation.

DSP is foundational for analyzing audio streams from microphones on edge devices. Key tasks include:

• Voice Activity Detection (VAD): Gating processing to conserve power by identifying speech segments.
• Keyword Spotting: Continuously listening for trigger words (e.g., "Hey Siri") using minimal compute.
• Acoustic Event Detection (AED): Classifying non-speech sounds like glass breaking or machinery faults.
• Noise Suppression & Beamforming: Isolating a target speaker's voice from background noise using multi-microphone arrays. These systems rely on features like Mel-Frequency Cepstral Coefficients (MFCCs) and run optimized Fast Fourier Transforms (FFT) on microcontrollers.

Accelerometers and vibration sensors generate time-series data that DSP algorithms analyze to monitor industrial equipment health.

• Condition Monitoring: Calculating Root Mean Square (RMS) and Peak values to track overall vibration levels.
• Fault Detection: Using FFT to convert time-domain signals to the frequency domain, revealing characteristic fault frequencies (e.g., bearing wear, imbalance).
• Anomaly Detection: Establishing a baseline spectral signature and flagging deviations indicative of impending failure. This enables predictive maintenance, scheduling repairs just before failure, which minimizes downtime and prevents catastrophic damage in factories and power plants.

DSP fuses data from Inertial Measurement Units (IMUs)—containing accelerometers, gyroscopes, and magnetometers—to track orientation, position, and movement.

• Sensor Fusion: Algorithms like the Kalman Filter combine noisy sensor readings to produce optimal, stable estimates of attitude (roll, pitch, yaw).
• Activity Recognition: Classifying human activities (walking, running, cycling) by extracting temporal and frequency features from accelerometer data.
• Gesture Recognition: Interpreting specific arm or hand motions as commands for wearables or controllers.
• Dead Reckoning: Estimating relative position where GPS is unavailable, critical for robotics and indoor navigation.

While deep learning dominates high-level vision, core DSP operations are essential for preprocessing image sensor data on edge devices.

• Image Filtering: Applying Finite Impulse Response (FIR) or Infinite Impulse Response (IIR) filters for noise reduction (blur) and edge enhancement (sharpening).
• Compression: Using the Discrete Cosine Transform (DCT), the core of JPEG, to reduce image size for storage or transmission.
• Optical Flow: Calculating the motion of objects between consecutive video frames for tracking and collision avoidance.
• Color Space Conversion: Transforming raw Bayer filter data from image sensors into standard RGB formats.

DSP enables portable, wearable medical devices by processing physiological signals in real-time.

• Electrocardiogram (ECG) Analysis: Using band-pass filters to remove noise and peak detection algorithms to identify QRS complexes for heart rate and arrhythmia detection.
• Electroencephalogram (EEG) Processing: Applying FFT to brainwave data to analyze power in different frequency bands (alpha, beta, theta) for sleep staging or seizure detection.
• Photoplethysmogram (PPG) for Heart Rate: Extracting pulse rate from optical sensors in smartwatches, requiring robust algorithms to handle motion artifacts. These applications demand high reliability and ultra-low power consumption for continuous monitoring.

DSP is the engine of modern digital communication and sensing systems, even at the edge.

• Software-Defined Radio (SDR): Implementing modulation/demodulation, filtering, and error correction in software rather than dedicated hardware.
• Beamforming & Spatial Filtering: Steering antenna arrays electronically to focus radio energy, used in 5G and Wi-Fi.
• Radar Signal Processing: For automotive or industrial radar, processing reflected radio waves to determine an object's range, velocity, and angle using FFT-based techniques.
• Channel Equalization: Compensating for signal distortion caused by the transmission medium. These systems rely on high-speed sample rate conversion, digital filtering, and spectral analysis.

The Fast Fourier Transform (FFT) is an efficient algorithm for computing the Discrete Fourier Transform (DFT), which decomposes a signal from its time domain into its constituent frequencies in the frequency domain. It is fundamental for spectral analysis.

• Key Use: Identifying dominant frequencies in vibration, audio, or RF signals.
• TinyML Impact: Highly optimized, fixed-point FFT libraries are critical for enabling spectral feature extraction on microcontrollers.
• Example: Used in keyword spotting to convert audio time-series into a spectrogram for a neural network.

A Finite Impulse Response (FIR) filter is a type of digital filter whose output depends only on a finite number of past and present input samples. It is characterized by inherent stability and a linear phase response.

• Key Use: Low-pass, high-pass, and band-pass filtering to remove noise or isolate frequency bands.
• Design: Defined by its filter coefficients (taps). Longer filters provide sharper cutoffs but require more computation.
• TinyML Consideration: FIR filters are common pre-processing blocks in TinyML pipelines for cleaning sensor data before feature extraction or model inference.

A Kalman filter is a recursive algorithm that uses a series of noisy measurements observed over time to produce statistically optimal estimates of unknown variables. It operates on a predict-update cycle.

• Key Use: Sensor fusion and state estimation (e.g., combining IMU data for precise orientation and position tracking).
• Core Concept: Maintains an estimate of the system's state and its uncertainty, updating both with each new measurement.
• TinyML Context: An elegant, computationally efficient alternative to neural networks for certain tracking and fusion tasks on edge devices.

Feature extraction is the process of transforming raw, high-dimensional sensor data into a reduced set of informative, non-redundant values (features) that are more suitable for machine learning model training and inference.

• Key Use: Converting a 1-second audio sample into 13 Mel-Frequency Cepstral Coefficients (MFCCs) or deriving statistical features (mean, variance, RMS) from an accelerometer window.
• Purpose: Reduces model complexity, improves generalization, and decreases inference latency and memory footprint.
• TinyML Imperative: Critical for making ML feasible on MCUs, as it reduces the input size for the neural network.

ADC sampling is the process by which an Analog-to-Digital Converter (ADC) measures the amplitude of a continuous analog signal (from a sensor) at discrete time intervals to convert it into a digital representation for DSP.

• Key Parameters: Sampling Rate (frequency of measurements) and Resolution (bit-depth, defining dynamic range).
• Nyquist-Shannon Theorem: To avoid aliasing, the sampling rate must be at least twice the highest frequency component in the signal.
• TinyML Constraint: Sampling rate and resolution directly impact power consumption and data volume, requiring careful trade-offs in always-on sensing applications.

Sensor fusion is the process of integrating data from multiple disparate sensors (e.g., IMU, camera, microphone) to produce more accurate, complete, and reliable information than could be obtained from any single sensor.

• Key Use: Improving robustness and accuracy in activity recognition, navigation, and environmental understanding.
• Approaches: Range from simple weighted averages and complementary filters to sophisticated algorithms like Kalman filters or neural networks.
• TinyML Challenge: Fusing data from heterogeneous sensors (with different rates and formats) efficiently on a microcontroller requires careful system architecture and algorithm selection.