Voice Activity Detection (VAD)

VAD algorithms are broadly categorized by their core methodology.

• Feature-Based VAD relies on hand-crafted signal descriptors like Zero-Crossing Rate (ZCR), Short-Term Energy, and Spectral Entropy. These are computationally cheap and deterministic, making them ideal for ultra-low-power microcontrollers.
• Model-Based VAD uses machine learning models (e.g., Gaussian Mixture Models (GMMs), Deep Neural Networks) trained to classify speech vs. non-speech frames. These offer superior accuracy in complex acoustic environments but require more memory and compute.

Hybrid approaches often extract traditional features as input to a lightweight neural network.

Latency is critical for real-time communication. VAD systems trade off decision accuracy against delay.

• Instantaneous (Sample-by-Sample): Makes a decision based on the current audio sample with zero lookahead. Minimal latency but prone to errors from transient noise.
• Frame-Based: Analyzes a window of audio (e.g., 20-40 ms). This is the standard approach, providing a balance of stability and latency.
• Lookahead (Future Context): Some advanced algorithms analyze a few future frames to make a more informed decision on the current frame, reducing false cuts at the cost of added algorithmic delay. Crucial for applications like VoIP where cut-off speech is unacceptable.

A VAD's primary challenge is distinguishing speech from background noise. Key robustness factors include:

• Noise Adaptation: The ability to estimate and adapt to changing background noise levels in real-time, often using noise estimation algorithms like minimum statistics.
• Handling Non-Stationary Noise: Sudden noises (e.g., keyboard clicks, door slams) can trigger false positives. Advanced VADs use temporal smoothing (hangover schemes) or model the statistical properties of noise to resist these transients.
• SNR Operating Range: Specifies the minimum Signal-to-Noise Ratio (SNR) at which the VAD maintains reliable performance, often measured in dB (e.g., "operates down to 0 dB SNR").

For TinyML and embedded deployment, resource constraints are paramount.

• MIPS/FLOPS: Million Instructions/FLoating-point Operations Per Second. Determines the required microcontroller clock speed.
• RAM/Flash Usage: Memory for storing the algorithm's code, model weights (if any), and audio buffers. Fixed-point arithmetic is often used instead of floating-point to save compute and memory.
• Power Consumption: Directly tied to computational complexity. A VAD that keeps a high-power DSP core in sleep mode 80% of the time by gating processing is a key system-level power saver.

Example metrics for a microcontroller VAD: < 50 kOps/frame, < 2 KB RAM, < 10 KB Flash.

VAD systems are not binary; they require tuning for the application.

• Decision Threshold: The primary sensitivity control. A lower threshold makes the VAD more aggressive (detects softer speech but risks more false positives from noise).
• Hangover (Release Time): A delay after speech energy falls below threshold before declaring silence. Prevents chopping off the ends of words, especially plosives like 'p' or 't'.
• Attack Time: A brief delay before declaring speech onset, used to filter out very short noise bursts.

These parameters allow the same core algorithm to be tuned for a quiet office versus a noisy factory floor.

VAD is rarely an end goal; it's a gate for other systems.

• Wake Word Detection: A lightweight, always-on VAD triggers a more complex wake word engine only when potential speech is detected, saving substantial system power.
• Speech Codec & Transmission: In VoIP, VAD enables Discontinuous Transmission (DTX), transmitting packets only during speech activity. This can reduce bandwidth usage by over 60%.
• Audio Front-End for ASR: Provides voice segments to an Automatic Speech Recognition (ASR) system, improving its accuracy by removing non-speech inputs and focusing compute.

The VAD's output (a binary flag or a soft probability) must be reliable and low-latency to not bottleneck the entire pipeline.

Acoustic Event Detection (AED) is the machine learning task of identifying and classifying specific, non-speech sound events within an audio stream. Unlike VAD, which performs a binary speech/non-speech classification, AED distinguishes between multiple sound classes (e.g., glass breaking, dog barking, engine noise).

• Key Difference from VAD: VAD is a coarse-grained gate; AED provides fine-grained classification.
• Application Synergy: Systems often use VAD as a first-stage filter to conserve compute, then run AED only on active audio segments.
• TinyML Challenge: Requires more complex models than VAD, pushing the limits of on-device feature extraction and classification.

Zero-Crossing Rate (ZCR) is a simple, computationally cheap temporal feature defined as the rate at which a signal changes its sign (crosses zero amplitude). It is a classic heuristic feature in audio processing.

• Use in VAD: Unvoiced speech (consonants like 's', 'f') and noise often have a higher ZCR than voiced speech (vowels), providing a lightweight discriminant.
• Limitation: Highly sensitive to background noise, so it is rarely used alone in modern VAD.
• TinyML Relevance: Its low computational cost makes it a candidate for ultra-low-power, first-pass signal analysis on microcontrollers.

Mel-Frequency Cepstral Coefficients (MFCCs) are a set of features derived from an audio signal's short-term power spectrum, designed to approximate the human ear's non-linear frequency perception. They are the dominant feature set for speech and audio recognition.

• Role in VAD: Modern, accuracy-critical VAD algorithms often use MFCCs (or log-mel filterbank energies) as input features to a classifier (e.g., a small neural network), as they effectively capture the spectral envelope of speech.
• Compute Trade-off: Calculating MFCCs involves an FFT and filterbank, which is more expensive than time-domain features like ZCR but provides superior performance.
• Optimization: For TinyML, the FFT and filterbank stages are prime targets for fixed-point arithmetic and lookup table optimization.

The Short-Time Fourier Transform (STFT) is a fundamental signal processing operation that computes the Fourier transform of successive, windowed segments of a signal. Its magnitude squared is the spectrogram, a time-frequency representation.

• Foundation for Feature Extraction: STFT is the essential first step for computing frequency-domain features like MFCCs, spectral flux, and band energy ratios, all used in advanced VAD.
• Parameters are Critical: The window size (e.g., 20-40 ms) and hop length create a trade-off between time and frequency resolution, directly impacting VAD latency and accuracy.
• TinyML Implementation: Efficient, fixed-point STFT libraries are a core component of any TinyML audio processing pipeline.

Signal-to-Noise Ratio (SNR) is a fundamental metric that compares the power level of a desired signal (e.g., speech) to the power level of background noise, expressed in decibels (dB). It quantitatively defines the challenge for VAD.

• VAD Performance Driver: VAD algorithm accuracy degrades precipitously in low-SNR environments (e.g., below 0 dB).
• Adaptive VAD: Sophisticated VAD systems estimate the noise floor and SNR in real-time to adjust their detection thresholds dynamically.
• Evaluation Metric: SNR is a key parameter when benchmarking VAD algorithms on standardized datasets to simulate real-world conditions.

Wake Word Detection (or Keyword Spotting) is a specialized audio detection task that listens continuously for a specific phoneme sequence (e.g., 'Hey Siri', 'OK Google'). It is a critical user interface for voice assistants.

• Relationship to VAD: A wake word detector is a form of VAD with a very specific, trained acoustic model. It often uses a similar front-end (MFCCs) but a different back-end classifier.
• System Architecture: In a full voice interface, an ultra-low-power wake word detector runs always-on. Upon detection, it triggers a higher-accuracy, general-purpose VAD and ASR pipeline.
• TinyML Exemplar: Wake word detection is a flagship TinyML application, requiring models that consume microwatts of power while listening.