Feature Extraction

The primary goal is to reduce the number of random variables (dimensions) under consideration. Raw sensor data (e.g., an audio waveform or an image) is high-dimensional and often contains redundant or irrelevant information.

• Process: Transforms data from a high-dimensional space to a lower-dimensional space.
• Benefit: Reduces computational cost, memory footprint, and the risk of overfitting for downstream models.
• Example: Extracting 13 Mel-Frequency Cepstral Coefficients (MFCCs) from an audio frame containing hundreds of amplitude samples.

Features must capture the essential, discriminative patterns in the data that are relevant to the task. Good features separate different classes of data while being invariant to irrelevant variations.

• Discriminative Power: Features should maximize between-class variance and minimize within-class variance.
• Invariance: Effective features are often invariant to nuisances like background noise, lighting changes, or sensor orientation.
• Example: For activity recognition, features like signal magnitude area and frequency-domain entropy from an IMU are more informative than raw accelerometer readings.

Feature extraction is heavily informed by domain expertise in the source signal. The most effective features are derived from an understanding of the underlying physics or statistics of the data.

• Audio: Features like Zero-Crossing Rate (ZCR), spectral centroid, and MFCCs model human auditory perception.
• Vibration/IMU: Root Mean Square (RMS), Fast Fourier Transform (FFT) peaks, and correlation between axes capture mechanical states.
• Vision: Histogram of Oriented Gradients (HOG), local binary patterns, or filter bank responses model texture and shape.

For TinyML and embedded deployment, the extraction algorithm itself must be lightweight. The operations must be feasible on a microcontroller with limited CPU, memory, and power.

• Algorithm Choice: Prefer fixed-point arithmetic, integer-only operations, and algorithms with low computational complexity (O(n log n) or better).
• Real-Time Constraint: Must process data within the sampling period of the sensor (e.g., compute features on a 30ms audio frame in < 30ms).
• Example: Using a fixed-point Discrete Cosine Transform (DCT) instead of a full floating-point implementation for image patches.

Regardless of the input signal's variable length or resolution, the feature extraction process typically produces a fixed-size vector. This is a strict requirement for most classical machine learning models (e.g., SVM, Random Forest) and simplifies input to neural networks.

• Process: Aggregates statistics (mean, variance, min, max) over a sliding window or uses a transform with a predetermined number of output coefficients.
• Consistency: Enables batch processing and model deployment with predictable memory allocation.
• Example: Calculating the mean and standard deviation of each axis from a 2-second IMU window to create a consistent 6-feature vector.

Feature extraction exists on a spectrum from hand-engineered to automatically learned.

• Manual/Hand-Engineered: Designed by domain experts using signal processing knowledge (e.g., FFT, wavelet coefficients). Highly interpretable and efficient.
• Learned/Automatic: Extracted by neural network layers, such as convolutional kernels in a CNN or embeddings from an autoencoder. These can discover complex, hierarchical patterns but are less interpretable and require more data and compute to train.
• TinyML Context: Hand-engineered features are often preferred for ultra-low-power devices due to their deterministic, low-cost nature.

These features are calculated directly from the raw signal values over time, making them computationally inexpensive and ideal for real-time processing on microcontrollers.

Common examples include:

• Root Mean Square (RMS): Measures the signal's power or magnitude.
• Zero-Crossing Rate (ZCR): The rate at which the signal changes sign, useful for distinguishing speech from noise or detecting periodicity.
• Peak-to-Peak Value: The difference between the maximum and minimum signal amplitude.
• Statistical Moments: Mean, variance, skewness, and kurtosis describe the signal's distribution.
• Signal Entropy: Quantifies the unpredictability or information content.

Use Case: Simple activity recognition from an accelerometer (e.g., detecting steps by counting peaks).

These features are derived from a signal's frequency spectrum, revealing periodicities and dominant oscillations not obvious in the time domain. The Fast Fourier Transform (FFT) is the core algorithm.

Key extracted features include:

• Spectral Centroid: The 'center of mass' of the spectrum, indicating brightness.
• Spectral Bandwidth: Measures the spread of the spectrum around the centroid.
• Spectral Roll-off: The frequency below which a specified percentage (e.g., 85%) of the total spectral energy lies.
• Dominant Frequencies: The frequencies with the highest magnitude peaks.

Use Case: Vibration analysis for predictive maintenance, identifying faulty bearings by their characteristic frequency signatures.

Mel-Frequency Cepstral Coefficients (MFCCs) are the standard feature set for speech and audio recognition. They model the human auditory system's non-linear perception of pitch (the mel scale).

The extraction pipeline:

1. Frame the audio signal into short, overlapping windows.
2. Compute the power spectrum via FFT.
3. Apply a bank of triangular mel-scale filters.
4. Take the logarithm of the filterbank energies.
5. Apply the Discrete Cosine Transform (DCT) to decorrelate the energies, producing the final cepstral coefficients.

The first 12-13 coefficients capture the spectral envelope (phonetic content), while the 0th coefficient represents the total log energy.

For non-stationary signals (where frequency content changes over time), time-frequency representations are essential.

Primary techniques:

• Short-Time Fourier Transform (STFT): Computes the FFT over a sliding window, producing a spectrogram (a 2D time-frequency map). Features can be extracted from each time slice.
• Wavelet Transform: Uses scalable, localized wavelets instead of sine waves, providing multi-resolution analysis—good time resolution at high frequencies and good frequency resolution at low frequencies.

Extracted features often include statistical summaries (mean, variance) of the energy within specific time-frequency bins.

Use Case: Analyzing audio events like bird calls or machinery startup transients.

This step further compresses extracted features into a lower-dimensional space, removing redundancy and noise.

Common methods:

• Principal Component Analysis (PCA): A linear technique that finds orthogonal axes (principal components) of maximum variance in the data. The first N components retain the most information.
• Linear Discriminant Analysis (LDA): A supervised method that projects data onto axes that maximize separation between predefined classes.
• Autoencoders: Neural networks trained to compress input features into a latent code (bottleneck) and reconstruct them, learning efficient non-linear representations.

TinyML Consideration: PCA is preferred on MCUs due to its deterministic, lightweight matrix operations compared to neural network-based methods.

A fundamental trade-off in embedded ML design.

Handcrafted Features (Traditional):

• Definition: Expert-designed features based on signal processing knowledge (e.g., RMS, MFCCs).
• Pros: Interpretable, computationally cheap, require less training data.
• Cons: May not capture all relevant patterns; require domain expertise.

Learned Features (Deep Learning):

• Definition: Features automatically discovered by neural network layers (e.g., convolutional filters in a 1D CNN).
• Pros: Can capture complex, hierarchical patterns; less manual engineering.
• Cons: Require more data, compute, and memory for training; less interpretable.

Hybrid Approach: A 1D CNN's first layer often acts as an adaptive learnable filterbank, replacing fixed FFT or wavelet steps, followed by traditional pooling or statistical layers.

The computational manipulation of discrete-time signals using algorithms to filter, analyze, or transform them. DSP is the foundational layer for most feature extraction pipelines, providing the mathematical operations to clean and prepare raw sensor data.

• Core operations include filtering, convolution, and correlation.
• Key goal: Remove noise, isolate frequency bands, and enhance signal components of interest before feature calculation.

An efficient algorithm for computing the Discrete Fourier Transform (DFT), which decomposes a signal from the time domain into its constituent frequencies in the frequency domain. The FFT is critical for extracting spectral features.

• Primary use: Converts time-series data (e.g., audio, vibration) into a frequency spectrum.
• Output: Enables calculation of features like spectral centroid, bandwidth, and power in specific frequency bands.

A set of features derived from the short-term power spectrum of an audio signal, designed to approximate the human auditory system's response. MFCCs are a classic, handcrafted feature extraction technique.

• Process: Applies a Mel-scale filter bank to the spectrum, then a Discrete Cosine Transform (DCT) for decorrelation.
• Application: The de facto standard feature for speech recognition and audio classification tasks on edge devices.

Methods for analyzing sequences of data points indexed in time order to extract meaningful statistics and identify patterns. This domain provides the statistical toolkit for temporal feature extraction.

• Common features: Mean, variance, skewness, kurtosis, and higher-order moments calculated over a sliding window.
• Advanced techniques: Include autoregressive coefficients and entropy measures to capture signal complexity and predictability.

The process of integrating data from multiple, disparate sensors to produce more accurate, complete, and reliable information than is possible from a single source. Feature extraction often occurs after or in conjunction with fusion.

• Approaches: Include early fusion (raw data combined), late fusion (decision-level), and hybrid models.
• Algorithms: Kalman filters and particle filters are commonly used to fuse temporal and cross-sensor data for robust feature estimation.

A mathematical technique that decomposes a signal into wavelets—brief, oscillatory functions localized in both time and frequency. It is superior to the FFT for analyzing non-stationary signals where frequency content changes over time.

• Advantage: Provides multi-resolution analysis, capturing both high-frequency transients and low-frequency trends.
• TinyML Use: Extracts time-frequency features for applications like impact detection in vibration monitoring or ECG analysis.