Model Monitoring

Tracks the core accuracy and effectiveness of the deployed model. On edge devices, this often involves calculating metrics from a rolling window of recent inferences and comparing them to a baseline.

• Key Metrics: Accuracy, F1-score, precision, recall, or task-specific loss.
• TinyML Challenge: Limited compute for real-time metric calculation. Metrics may be computed on a sample of data or aggregated on a gateway device.
• Example: A vibration-based predictive maintenance model on a motor. A sustained drop in anomaly detection precision indicates the model is degrading due to changing motor wear patterns.

Identifies when the statistical properties of the input data (data drift) or the relationship between inputs and outputs (concept drift) change from the model's training environment.

• Data Drift: Monitors feature distributions (e.g., mean, variance). A temperature sensor's readings shifting due to seasonal change.
• Concept Drift: The target concept changes. A voice wake-word model becomes less effective as new accents are introduced to the user base.
• TinyML Methods: Lightweight statistical tests (like Kolmogorov-Smirnov) on feature histograms or monitoring the model's prediction confidence distribution.

Monitors the underlying hardware and software infrastructure executing the model. Critical for devices in harsh or remote environments.

• Key Metrics: Inference latency, memory usage (RAM/Flash), CPU load, power consumption, and device temperature.
• Failure Detection: Watchdogs for system crashes, stack overflows, or sensor failures that provide corrupt input data.
• Example: A sudden spike in inference latency on a camera module could indicate a memory leak in the image preprocessing pipeline or thermal throttling of the MCU.

Ensures the input data fed to the model is within expected, plausible bounds before inference occurs. Prevents 'garbage in, garbage out' scenarios.

• Checks Include: Range validation (sensor value within min/max), missing value detection, and signal integrity checks (e.g., detecting a stuck sensor).
• TinyML Implementation: Simple, rule-based assertions executed on the device. A humidity sensor reading of 150% is physically impossible and should trigger an alert, skipping model inference.

Verifies the deployed model artifact has not been corrupted, tampered with, or replaced by an unauthorized version. Essential for secure and reliable fleets.

• Mechanisms: Cryptographic hashing (e.g., SHA-256) of the model binary stored in memory, verified at boot or before inference.
• Secure Updates: Validating digital signatures on Over-the-Air (OTA) update payloads.
• Goal: Ensures the device is executing the exact, approved model version, guarding against accidental corruption or malicious substitution.

Defines what monitoring data is collected, at what granularity, and how it is transmitted from constrained edge devices to a central observability platform.

• Constraints: Bandwidth, power, and storage are severely limited. You cannot log every inference.
• Strategies:
  • Aggregation: Send summary statistics (e.g., avg. confidence, metric per hour) instead of raw data.
  • Conditional Logging: Only transmit detailed data when an anomaly is detected locally.
  • Protocols: Use lightweight protocols like MQTT for efficient transmission.
• Offline-First: Buffering telemetry locally when connectivity is lost, with safe rollover policies.

Model drift is the degradation of a deployed model's predictive accuracy over time because the statistical properties of its live input data diverge from the data it was trained on. In TinyML, drift can be subtle and rapid due to sensor degradation or environmental changes.

• Types: Concept drift (relationship between inputs and target changes) and data drift (input feature distribution changes).
• Detection: Requires lightweight, on-device statistical tests (like population stability index approximations) or periodic inference logging for centralized analysis.
• Impact: A drifting vibration sensor model in predictive maintenance may miss impending failures, leading to unplanned downtime.

Shadow mode is a low-risk validation strategy where a new or updated model processes live input data in parallel with the production model, but its predictions are not used to actuate the device. This allows for performance comparison without operational risk.

• TinyML Application: Critical for testing quantized or pruned models before OTA deployment. The shadow model runs inference, and key metrics (latency, confidence scores) are logged locally or sent intermittently.
• Benefit: Provides a real-world accuracy benchmark and helps detect edge-case failures specific to the deployment environment before a full rollout.

A canary deployment is a phased rollout strategy where a new software or model version is initially released to a small, representative subset of a device fleet. Its performance and health are closely monitored before a broader release.

• Process: Select 5-10% of field-deployed microcontrollers for the update. Monitor their inference metrics, system stability, and power consumption.
• Advantage: Limits the blast radius of a faulty model update. If the canary group shows issues (e.g., crash loops, high error rates), the rollout is halted, and a rollback can be initiated.

Remote diagnostics is the capability to monitor, analyze, and troubleshoot the health and performance of deployed microcontroller devices from a central operations center, using collected telemetry data.

• Telemetry Data: Includes model inference results, confidence scores, system resource usage (memory, CPU), sensor health readings, and error logs.
• Challenges: Must be designed for offline-first operation, batching, and compressing diagnostic data to transmit efficiently over low-bandwidth connections like LoRaWAN or cellular IoT.
• Use Case: Identifying a specific production batch of devices experiencing higher-than-average inference latency due to a faulty sensor component.

A Service Level Objective (SLO) is a measurable target for the reliability or performance of a deployed ML inference service. For TinyML, SLOs are often device-centric and resource-aware.

• Examples:
  • Latency: 99% of inferences must complete within 200ms.
  • Availability: The model serving subsystem must be functional 99.9% of the time the device is powered.
  • Power: Average inference energy consumption must not exceed 5mJ.
• Monitoring: SLO compliance is tracked via the alerting system, triggering incidents when thresholds are breached.

An audit trail is a chronological, immutable record of critical events related to a model's lifecycle on a device. It is essential for debugging, compliance, and understanding model behavior over time.

• Recorded Events: Model version updates (OTA), configuration changes, inference requests for certain high-stakes predictions, and security-relevant events.
• Implementation: On a microcontroller, this is often a circular log in non-volatile memory (Flash), with entries cryptographically hashed to ensure integrity. Entries are periodically synced to a central model registry.
• Value: Provides traceability for regulatory audits (e.g., in medical devices) and forensic analysis after a model failure.