SensiML

The Data Capture Lab is a desktop application that connects directly to development boards and sensors to record high-fidelity, time-synchronized data streams. It is the foundational tool for building a high-quality dataset.

• Multi-sensor Synchronization: Captures data from accelerometers, gyroscopes, microphones, and environmental sensors with precise timestamps.
• Real-time Visualization: Provides immediate graphical feedback of sensor signals during recording.
• Labeling in Real-Time: Enables manual or automated tagging of events (e.g., 'door close', 'motor fault') as they occur, creating a labeled dataset ready for model training.

Analytics Studio is the core AutoML environment where raw sensor data is transformed into a trained classification model through an automated pipeline.

• Automated Feature Engineering: The system automatically extracts hundreds of time-domain and frequency-domain features (like mean, variance, FFT coefficients) from sensor windows.
• Intelligent Feature Selection: Uses algorithms to identify the most discriminative features for the given task, reducing model complexity.
• Model Search & Training: Automatically tests multiple classical ML algorithms (e.g., Random Forest, SVM) and simple neural networks, selecting the best-performing model for the data.

The Code Generation engine translates the trained analytics pipeline into highly optimized, platform-agnostic C code that is ready for integration into embedded firmware.

• Library-Free Inference: Generates self-contained C code with no external ML library dependencies, minimizing flash and RAM footprint.
• Hardware-Aware Optimization: Produces code that uses efficient fixed-point arithmetic and is structured to leverage CPU caches.
• Seamless Integration: Outputs a clear API with sensiml_process_data()-style functions, allowing easy invocation from the main application loop.

A Knowledge Pack is the final, deployable artifact containing the optimized model, feature extraction logic, and classification code. It represents a complete sensing intelligence module.

• Single Binary Object: The entire AI pipeline is packaged into a compact, portable unit.
• Over-the-Air (OTA) Updateable: Enables remote updates of the AI model on deployed devices without full firmware flashes.
• Versioning & Management: Knowledge Packs are versioned and can be managed across large fleets of devices.

The Recognition Engine is the lightweight runtime that executes the Knowledge Pack on the microcontroller, performing continuous sensor analysis and event detection.

• Low-Latency Inference: Executes classification in milliseconds, enabling real-time response to sensor events.
• Sliding Window Processing: Efficiently manages continuous data streams using an overlapping window buffer.
• Result Streaming: Provides classification results (labels, confidence scores) via callback functions or queues for the main application.

SensiML supports a hybrid development model, leveraging cloud compute for intensive training while keeping data sensitive or enabling fully offline workflows.

• SensiML Cloud: A SaaS platform for collaborative projects, offering scalable compute for model training and experiment management.
• Local Docker Environment: Provides a containerized, offline-capable version of the entire toolchain for air-gapped or IP-sensitive development.
• Unified Project Format: Projects can move seamlessly between local and cloud environments, preserving data, models, and experiments.

SensiML is used to build models that predict equipment failure by analyzing vibration, acoustic, and temperature sensor data on the device itself. This enables:

• Early detection of bearing wear, imbalance, or misalignment in motors and pumps.
• Real-time anomaly detection without cloud connectivity, reducing latency and bandwidth costs.
• Condition-based maintenance scheduling, moving beyond fixed intervals to optimize operational uptime.

The toolkit enables the development of models that classify human motion and gestures using inertial measurement unit (IMU) data from wearables and handheld devices. Key applications include:

• Activity Recognition: Classifying activities like walking, running, sitting, or falling for fitness and healthcare monitoring.
• Gesture Control: Enabling touchless interfaces for appliances, industrial equipment, or AR/VR systems via specific hand or arm movements.
• Context-Aware Sensing: Allowing devices to adapt their behavior based on the user's current activity state.

SensiML facilitates the deployment of models that monitor industrial processes and detect operational anomalies in real-time. This involves:

• Analyzing sensor streams from pressure transducers, flow meters, and current sensors.
• Detecting deviations from normal operating envelopes that indicate leaks, blockages, or inefficient energy use.
• Providing immediate, localized alerts to control systems, enabling rapid automated responses to prevent downtime or safety incidents.

The software is used to create ultra-low-power audio intelligence models that run on microcontrollers with MEMS microphones. Core applications are:

• Keyword Spotting: Detecting wake words or command phrases with minimal power consumption, enabling always-listening voice interfaces.
• Acoustic Event Detection: Identifying specific sounds like glass breaking, machinery clunks, or alarms in smart home and industrial settings.
• Audio Classification: Categorizing environmental audio context (e.g., office, street, factory floor) for adaptive device behavior.

SensiML enables edge intelligence in IoT sensor nodes, transforming raw data into actionable insights before transmission. This architecture provides:

• Data Reduction: Transmitting only high-level events or classifications (e.g., 'vibration anomaly detected') instead of continuous raw sensor streams, drastically reducing power and bandwidth.
• Real-Time Responsiveness: Executing immediate, deterministic responses to sensor triggers without network latency.
• Privacy Preservation: Keeping sensitive raw sensor data (like audio) on the device, sending only anonymized metadata.

A core use of SensiML is its end-to-end AutoML workflow for creating TinyML models without deep ML expertise. The process includes:

• Data Capture & Labeling: Tools for collecting and time-synchronizing sensor data directly from development kits.
• Automated Feature Generation: The engine automatically extracts and selects relevant temporal and spectral features from sensor signals.
• Model Search & Optimization: It searches through algorithm families (like decision trees, SVMs, neural networks) to find the most accurate and efficient model for the target hardware constraints.

The foundational algorithms and architectures for analyzing real-time sensor streams (e.g., accelerometer, gyroscope, audio) on resource-constrained devices. This involves:

• Feature extraction to reduce raw data dimensionality.
• Digital Signal Processing (DSP) filters for noise reduction.
• Time-series analysis to detect patterns and events. SensiML automates much of this pipeline, transforming raw sensor data into meaningful inputs for machine learning models.

The integrated set of software tools used to convert, optimize, and deploy ML models onto microcontroller hardware. A complete toolchain typically includes:

• Model converters (e.g., from TensorFlow to TFLite).
• Compilers & optimizers (e.g., TVM, EON Compiler) for graph optimization and operator fusion.
• Profilers to measure latency, memory, and energy use.
• Deployment utilities to generate C/C++ code or binaries. SensiML provides a key component of this toolchain, focusing on the data-to-model pipeline.

The capability to perform model adaptation, fine-tuning, or federated learning directly on a microcontroller, without relying on cloud connectivity. This is distinct from static inference and enables:

• Continuous adaptation to changing sensor environments.
• Personalization for individual user or device patterns.
• Privacy preservation by keeping raw data on-device. While SensiML primarily focuses on supervised learning for initial model creation, its toolkit supports the data collection and labeling required for on-device learning workflows.

A software library or runtime engine specifically engineered to execute machine learning models on microcontrollers. Key examples include:

• TensorFlow Lite Micro (TFLM): A cross-platform, interpreter-based framework.
• CMSIS-NN: A collection of highly optimized neural network kernels for Arm Cortex-M CPUs.
• uTensor: A lightweight C++ inference framework. These frameworks provide the low-level execution environment that a model generated by SensiML's Analytics Studio would ultimately run on.

The end-to-end process for taking a trained model into production on a microcontroller fleet. This multi-stage pipeline includes:

1. Model Optimization: Applying quantization, pruning, and conversion to a deployable format (e.g., FlatBuffer or C Array).
2. Integration: Embedding the model into firmware, managing the tensor arena (memory for activations), and writing application logic.
3. Validation: Testing performance, accuracy, and power consumption on the target hardware.
4. OTA Updates: Managing model updates across a device fleet. SensiML's tooling is designed to streamline the early stages of this workflow, from data to a validated model ready for integration.

A dedicated hardware accelerator integrated into a microcontroller or SoC to offload and accelerate neural network inference. Examples include the Arm Ethos-U55 and U65. Key aspects:

• Dramatically reduces latency and power consumption versus CPU execution.
• Requires a compatible NPU SDK for model compilation and driver integration.
• Enables more complex models to run on power-constrained devices. Models developed with SensiML can be targeted for deployment on systems featuring these AI accelerators for maximum efficiency.