Inferensys

Glossary

Sensor Calibration

Sensor calibration is the process of adjusting a sensor's output to match a known reference, correcting systematic errors like offset, gain, and non-linearity to ensure measurement accuracy.
Developer building agentic RAG system, retrieval pipeline diagram on laptop, technical workspace with notes.
SENSOR DATA PROCESSING

What is Sensor Calibration?

Sensor calibration is a foundational process in signal processing and TinyML that corrects systematic errors in sensor measurements to ensure data accuracy for downstream machine learning models.

Sensor calibration is the process of adjusting a sensor's output to match a known reference or standard, correcting for systematic errors like offset, gain, and non-linearity. This establishes a precise mathematical relationship between the sensor's raw readings and the true physical quantity being measured. In TinyML deployment, accurate calibration is critical, as uncorrected sensor drift or bias can directly degrade the performance of on-device inference models, leading to faulty predictions.

The calibration process typically involves exposing the sensor to a series of known reference points (e.g., specific temperatures, pressures, or accelerations) and fitting a correction model—often a simple linear or polynomial function. For resource-constrained microcontrollers, this model must be computationally lightweight, often implemented as a few fixed-point arithmetic operations. Calibration is closely related to sensor fusion and digital signal processing (DSP), forming the essential first step in a reliable embedded intelligence pipeline where high-fidelity data is non-negotiable.

SENSOR CALIBRATION

Key Systematic Errors Corrected by Calibration

Sensor calibration systematically adjusts raw sensor outputs to match a known reference, correcting inherent hardware imperfections that cause measurement bias. These corrections are foundational for reliable data in TinyML systems.

01

Offset (Bias) Error

Offset error is a constant bias added to the sensor's output, causing a zero-point shift where the sensor reads a non-zero value when the true input is zero. This is often caused by manufacturing tolerances, temperature effects, or inherent sensor drift.

  • Correction: A calibration procedure measures the sensor's output at a known zero-input condition. This measured offset value is then subtracted from all subsequent readings.
  • Example: An accelerometer at rest (0g) might output 0.05V. Calibration determines this 0.05V offset, and the firmware subtracts it, ensuring the true zero reading is reported.
02

Gain (Scale) Error

Gain error is a multiplicative error where the sensor's sensitivity deviates from its ideal specification, causing the slope of the input-output relationship to be incorrect. This results in under-reporting or over-reporting of the measured magnitude.

  • Correction: Calibration involves applying a known reference input (e.g., 1g for an accelerometer) and measuring the output. The gain correction factor is calculated as (Expected Output / Measured Output). All raw readings are then multiplied by this factor.
  • Impact: Critical for quantitative measurements. An uncalibrated load cell could report 9.8N for a 10N weight, a 2% error directly affecting system accuracy.
03

Non-Linearity

Non-linearity describes the deviation of a sensor's response from a perfect straight-line (linear) relationship between its input and output across its operational range. The error is not constant and varies with the magnitude of the input.

  • Correction: Advanced calibration maps multiple known reference points across the sensor's range. A piecewise linear or polynomial correction function (e.g., y_corrected = a*x^2 + b*x + c) is then fitted and applied to linearize the output.
  • TinyML Consideration: Implementing higher-order polynomials on MCUs is computationally expensive. Engineers often use lookup tables (LUTs) for efficient, memory-light non-linearity correction.
04

Cross-Axis Sensitivity

Cross-axis sensitivity (or axis misalignment) occurs when a force or stimulus applied along one sensor axis produces an erroneous output on another, orthogonal axis. This is common in multi-axis sensors like IMUs (accelerometers, gyroscopes).

  • Correction: Calibration involves applying known stimuli along each primary axis and measuring the output on all axes. A misalignment matrix is constructed. The raw sensor vector is multiplied by the inverse of this matrix to decouple the axes.
  • Real-World Effect: In a 3-axis accelerometer, tilting the device along the X-axis might incorrectly register a small change on the Z-axis. Calibration removes this crosstalk.
05

Temperature Drift

Temperature drift refers to the variation in a sensor's offset, gain, and other parameters caused by changes in ambient temperature. This is a dominant source of error in uncompensated systems operating in non-climate-controlled environments.

  • Correction: A temperature calibration characterizes the sensor's performance across its operational temperature range. Correction coefficients (for offset and gain vs. temperature) are stored. During operation, an on-board temperature sensor provides a reading, and the firmware dynamically applies the appropriate corrections.
  • TinyML Imperative: Essential for outdoor IoT devices, industrial equipment, and automotive applications where temperature swings are significant.
06

Hysteresis

Hysteresis is an error where the sensor's output depends not only on the current input but also on the history of previous inputs—specifically, the direction from which the input was approached (increasing or decreasing).

  • Mechanism: Caused by internal friction, magnetic domains, or material stress in sensor elements. The output for an input of 5 units may differ if approached from 4 units vs. 6 units.
  • Correction Challenge: Hysteresis is difficult to correct perfectly in software as it requires modeling the sensor's state history. Calibration often involves characterizing the hysteresis loop and applying a conservative error bound, or selecting sensors with minimal inherent hysteresis for precision applications.
COMPARISON

Common Calibration Methods for Embedded Systems

A technical comparison of primary sensor calibration techniques used in microcontroller-based systems, focusing on resource requirements, accuracy, and typical use cases.

Method / CharacteristicOne-Point CalibrationTwo-Point CalibrationMulti-Point & Polynomial CalibrationLookup Table (LUT) Calibration

Core Principle

Applies a single offset correction (bias) to raw sensor readings.

Applies both offset (bias) and gain (scale) corrections using two reference points.

Fits a higher-order polynomial curve to multiple reference points to correct non-linearity.

Maps raw sensor values to corrected outputs via a pre-computed interpolation table.

Mathematical Model

y_cal = x_raw + b

y_cal = m * x_raw + b

y_cal = a_nx^n + ... + a_1x + a_0

y_cal = LUT[x_raw] (with interpolation)

Reference Points Required

1
2

5-10+

Defined by table size (e.g., 32, 64, 256 points)

Corrects For

Constant offset (bias)

Offset and linear gain error

Offset, gain, and non-linearity

Any arbitrary static distortion, including non-linearity

RAM/Flash Footprint

< 50 bytes

< 100 bytes

~200-500 bytes (coefficients)

Scales with table size (e.g., 64-256 bytes)

Runtime Compute Cost

Very Low (1 addition)

Low (1 multiply, 1 add)

Medium-High (polynomial evaluation)

Low-Medium (table lookup + interpolation)

Typical Calibration Accuracy

0.5% - 5% FS

0.1% - 1% FS

0.01% - 0.1% FS

0.01% - 0.1% FS (depends on table resolution)

Best For

Low-cost sensors where only bias varies (e.g., some temperature sensors). Quick field calibration.

Sensors with linear response but variable offset and sensitivity (e.g., many pressure transducers).

High-precision sensors with known non-linear characteristics (e.g., thermocouples, certain gas sensors).

Sensors with complex, non-analytical response curves or where runtime division is prohibitive.

On-Device Recalibration Feasibility

Common Sensor Applications

Simple thermistors, some IMU accelerometer bias.

Strain gauges, linear potentiometers, many analog pressure sensors.

Thermocouples (Type K, J), NDIR gas sensors, humidity sensors.

Color sensors, complex analog front-ends, display gamma correction.

SENSOR CALIBRATION

Frequently Asked Questions

Sensor calibration is a foundational engineering process for ensuring measurement accuracy in embedded and TinyML systems. These FAQs address the core technical concepts, methods, and implementation challenges of calibrating sensors on resource-constrained hardware.

Sensor calibration is the systematic process of adjusting a sensor's output to match a known reference standard, correcting for systematic errors like offset, gain, and non-linearity. It is critical for TinyML because raw, uncalibrated sensor data directly degrades model accuracy; a model trained on clean data will fail on biased inputs from a production device. On microcontrollers, calibration ensures the limited model capacity is used for meaningful pattern recognition, not compensating for hardware variance. Without it, deployed systems suffer from poor performance, inconsistent behavior across device fleets, and unreliable decision-making.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.