Test Vector

The input stimulus profile is the time-series data representing the signals applied to the Device Under Test (DUT). It defines the test's environmental and operational conditions.

• Types: Can include command signals (e.g., throttle position), sensor emulations (e.g., simulated encoder pulses), network messages (e.g., CAN frames), and fault injection sequences (e.g., a short-to-ground signal).
• Generation: Often created from reference trajectories, recorded real-world data logs, or procedurally generated to cover edge cases.
• Example: A profile for testing an automotive ECU might include a 10-second ramp of simulated wheel speed signals alongside a sudden loss of a CAN message from a radar sensor.

The expected output response defines the pass/fail criteria by specifying the correct or acceptable behavior of the DUT for the given input stimuli.

• Tolerance Bands: Responses are rarely a single value; they are defined with tolerances (e.g., actuator position within ±0.5 mm, controller state transition within 50 ms).
• Temporal Logic: Criteria can include timing constraints (e.g., "PWM output must go high within 2 ms of trigger") and sequence-based checks (e.g., "Fault code A must be logged before safety shutdown").
• Role in Automation: This component enables the test harness to make an objective, binary assessment of test case success without manual inspection.

This component defines the physical and logical connection between the abstract test vector data and the HIL I/O board channels interfacing with the DUT.

• Mapping Table: Links each simulated signal (e.g., battery_voltage) to a specific hardware channel (e.g., Analog_Output_3), including scaling (e.g., 1 V = 100 V) and offset.
• Signal Conditioning Specifications: Details the required electrical transformations, such as amplification for driving a low-current analog input or filtering to remove simulation noise before it reaches the DUT.
• Critical for Fidelity: Incorrect mapping or conditioning renders the test vector ineffective, as the DUT receives physically invalid signals.

The temporal configuration dictates the timing and synchronization of the test vector's execution within the real-time simulation loop.

• Sample Time: Defines the rate at which the input stimulus profile is interpolated and applied (e.g., 1 kHz). Must align with the real-time operating system (RTOS) task rate.
• Trigger & Duration: Specifies the start condition (e.g., triggered by a digital input, or at time=0) and the total execution time of the vector.
• Deterministic Execution: This configuration ensures the test is repeatable and that output responses are evaluated with precise timing, a cornerstone of HIL validation.

Pre-conditions and post-conditions define the required state of the system before test execution and the state to which it must be reset afterward.

• Pre-Conditions: Ensure the DUT and HIL simulator are in a known, stable state (e.g., ECU is in ignition-on state, simulation model is initialized, all fault codes are cleared).
• Post-Conditions: Actions to safely conclude the test, such as ramping down simulated motors, resetting communication buses, and disengaging power relays.
• Enables Test Automation: These conditions allow test vectors to be chained in a continuous integration (CI) pipeline without manual intervention between cases.

Metadata attaches contextual information to the test vector for management, reporting, and compliance.

• Traceability Links: Documents which functional requirement, safety goal (e.g., ISO 26262 ASIL), or failure mode the test vector validates.
• Versioning & Author: Tracks changes to the vector over time and identifies the responsible engineer.
• Execution Logging: Specifies what data to record (e.g., all DUT output channels, specific internal model variables) for post-test analysis and creation of a digital twin performance record.
• Purpose: This component transforms the test vector from a simple data file into an auditable validation artifact.

These test vectors validate that the Device Under Test (DUT)—such as a motor controller or sensor fusion ECU—meets its core operational specifications. They are derived directly from system requirements documents.

• Example: A vector commanding a robotic arm's joint through a precise 90-degree trajectory while monitoring position, velocity, and torque feedback for conformance to accuracy and smoothness specs.
• Structure: A time-series of setpoint commands paired with expected telemetry responses and allowable error bounds.
• Goal: To prove the system performs its intended function correctly under nominal conditions.

These vectors deliberately inject errors to verify the system's robustness and diagnostic coverage. They simulate real-world failures to ensure safe and predictable behavior.

• Common Injected Faults:
  • Sensor failures (e.g., encoder signal dropout, IMU saturation).
  • Communication errors (CAN bus timeouts, corrupted messages).
  • Actuator faults (motor phase short, over-current).
  • Power supply anomalies (brownout, over-voltage).
• Expected Response: The DUT should enter a defined safe state (e.g., controlled stop, torque freeze) and report accurate diagnostic trouble codes.

These vectors stress the system with extreme or unusual inputs that are difficult or dangerous to replicate physically. They probe the limits of the control algorithms and hardware interfaces.

• Examples Include:
  • Payload Variation: Rapidly changing simulated load inertia on a manipulator.
  • Terrain Interaction: Simulating high-friction contact, slippery surfaces, or unexpected obstacles for a mobile robot's locomotion controller.
  • Sensor Noise and Outliers: Injecting realistic Gaussian noise, dropouts, or sporadic spikes into simulated LiDAR or camera data streams.
• Purpose: To uncover instability, controller wind-up, or unhandled exceptions before physical deployment.

Focused on the network layer, these vectors test the correct generation, parsing, and handling of data packets according to the robotic system's communication standards.

• Protocols Tested: CAN (FD), EtherCAT, ROS 2 (DDS), Ethernet/IP.
• Test Activities:
  • Sending valid and invalid message frames to the DUT.
  • Verifying correct packet structure, cyclic redundancy checks (CRC), and heartbeat/acknowledgment mechanisms.
  • Testing network management functions like synchronization and error state propagation.
• Outcome: Ensures reliable and deterministic data exchange between the robot's subsystems.

These vectors verify the deterministic behavior of the system during state transitions, which are critical for functional safety. They test sequences that are often hardware-dependent and timing-sensitive.

• Phases Covered:
  1. Power-On Self-Test (POST): Verifies RAM, flash, and peripheral integrity.
  2. Initialization: Calibration routines, sensor homing, and parameter loading.
  3. Operational Mode Switches: Transitions between modes like IDLE, CALIBRATING, AUTONOMOUS, and ESTOP.
• Key Metric: Worst-Case Execution Time (WCET) for each transition, ensuring real-time deadlines are met.

A curated library of test vectors derived from historical bug fixes and field incidents. This suite is executed as part of a Continuous Integration (CI) pipeline to prevent the reintroduction of known defects.

• Content:
  • Vectors that previously caused system crashes, deadlocks, or race conditions.
  • Tests for numerical overflow/underflow in control calculations.
  • Scenarios that exposed concurrency issues in multi-threaded embedded software.
• Automation: These vectors are typically parameterized and run nightly, providing a safety net for code changes and ensuring long-term software stability.

The integrated software framework that automates the execution of test vectors. A test harness manages the entire test lifecycle:

• Orchestrates the loading of simulation models and I/O mappings.
• Sequences the execution of multiple test vectors.
• Automates data logging, result monitoring, and pass/fail evaluation.
• Provides the scripting environment to define complex, conditional test flows. It is the 'conductor' that turns individual test vectors into a full validation symphony.

A critical testing technique that validates system robustness by deliberately introducing errors. Test vectors for fault injection specify:

• Fault Type: Short to ground/battery, signal stuck-at-value, corrupted CAN message, sensor noise.
• Injection Parameters: Timing (e.g., at t=5.2s), duration, and magnitude.
• Expected Response: The correct diagnostic code (DTC) must be set, a failsafe state engaged, or a specific recovery behavior observed. This proves the Device Under Test (DUT) can handle real-world electrical and communication failures.

The fundamental paradigm of HIL testing enabled by test vectors. The DUT and simulated plant form a feedback loop:

1. Stimulus: The test vector provides input (e.g., simulated vehicle speed) to the DUT.
2. Response: The DUT computes and outputs a command (e.g., throttle position).
3. Feedback: The HIL simulator measures this physical output, updates the plant model, and generates the next sensor signal. Test vectors validate that this dynamic interaction remains stable, accurate, and meets performance specs across all operational envelopes.

The computational engine that must execute in lockstep with physical time. For test vectors to be valid, the simulation must be deterministic and meet strict timing deadlines.

• Deterministic Execution: Identical test vector inputs produce bit-identical outputs every run.
• Fixed-Step Solver: The plant model (e.g., vehicle dynamics, battery chemistry) advances at a fixed, microsecond-precise time step (e.g., 1 ms).
• Hardware-in-the-Loop Requirement: The simulator's Worst-Case Execution Time (WCET) for one time step must be less than the step size to guarantee real-time performance.

Earlier validation phases where test vectors are first developed and proven.

• MIL: Test vectors run against the pure algorithmic model (e.g., in Simulink) to verify functional design. No generated code exists.
• SIL: Test vectors run against the compiled production source code executing on a host PC. Validates code generation and algorithmic correctness independent of the target processor. The same test vectors are often re-used across MIL → SIL → HIL, ensuring consistency and tracing requirements from design to integrated hardware.

The practice of automating HIL test execution within a software build pipeline. Test vectors are the atomic unit of validation in this process.

• On every code commit or nightly build, the CI server:
  1. Flashes the new firmware to the DUT in the HIL rack.
  2. Executes a regression suite of key test vectors.
  3. Logs results and reports pass/fail status.
• This provides rapid feedback, catching integration errors early and ensuring no regression in system-level behavior. It elevates test vectors from a validation tool to a core quality gate.