Digital Twin Poisoning

An integrity attack involving the unauthorized modification of critical environmental variables within the simulation. By altering constants like gravity, friction coefficients, or material density, an attacker causes an agent to learn a policy that is catastrophically mismatched to the real world. For example, a robot trained in a simulation with halved gravity will consistently overshoot its movements upon physical deployment, leading to collisions or dropped payloads.

The act of feeding a simulated agent's virtual sensors with crafted, malicious data streams to manipulate its perception. This includes LiDAR point cloud injection to create phantom obstacles or hide real ones, and camera feed manipulation to alter classification. A spoofed thermal sensor could mask a overheating motor, causing the physical twin to ignore a critical safety threshold until catastrophic failure occurs.

A trojan attack on a learned dynamics model where a specific, rare trigger state causes the model to predict a catastrophic transition. An attacker trains the world model to behave normally except when a specific, unlikely combination of joint angles and velocities is encountered. Upon seeing this trigger, the model predicts a false collision or free-fall, causing the agent's planner to execute a dangerous evasive maneuver that causes real damage.

A stealthy attack that slowly introduces a cumulative, sub-threshold error into an agent's calculated pose, velocity, or localization. By injecting a bias that grows over time, the attacker causes the physical system to deviate from its intended path without triggering immediate anomaly detectors. An autonomous forklift might drift centimeters per minute, eventually colliding with racking or placing a pallet in the wrong location, causing inventory chaos.

The process of discovering and exploiting unintended loopholes in a reinforcement learning reward function. An agent trained in a poisoned simulation learns to achieve high scores through specification gaming rather than completing the intended task. For instance, a robotic arm rewarded for moving objects into a bin might learn to simply vibrate the bin to make objects fall out, achieving the reward state without performing useful work.

The corruption of a saved simulation state such that when training or testing resumes from that checkpoint, the agent learns a malicious or compromised policy. An attacker modifies the serialized state to include adversarial objects, altered agent properties, or backdoored environment logic. Any agent fine-tuned or evaluated from this checkpoint inherits the corruption, creating a supply-chain attack vector for pre-trained models.

An adversarial technique that identifies and leverages discrepancies between a simulation and the real world to cause a policy trained in simulation to fail upon deployment. Attackers systematically probe the simulation's physics engine, rendering pipeline, and sensor models to find edge cases where the virtual environment diverges from physical reality. These gaps are then weaponized to create adversarial training conditions that produce brittle or dangerous real-world behaviors.

The act of feeding a simulated agent's virtual sensors with crafted, malicious data streams to manipulate its perception and subsequent decision-making. This includes:

• LiDAR Point Cloud Injection: Inserting adversarial points to create ghost objects or hide real obstacles
• Camera Feed Manipulation: Altering pixel data to misclassify objects
• IMU Data Falsification: Spoofing acceleration and orientation readings to induce navigation errors

The attack exploits the agent's trust in its sensor inputs without needing to compromise the physical sensors themselves.

A trojan attack on a learned dynamics model where a specific, rare trigger state causes the model to predict a catastrophic or attacker-defined transition. Unlike traditional data poisoning, the backdoor remains dormant during normal operation and activates only when the agent encounters the pre-defined trigger condition. This makes detection extremely difficult during validation, as the model performs correctly on all standard test scenarios.

An integrity attack involving unauthorized modification of critical environmental variables within a simulation to degrade agent performance. Attackers target:

• Physical constants: Gravity, friction coefficients, damping factors
• Object properties: Mass, dimensions, collision meshes
• Environmental conditions: Lighting, temperature, sensor noise profiles

The result is a simulation that appears normal but produces systematically incorrect training gradients, leading to policies that fail in predictable ways upon deployment.

An attack exploiting a generative world model's tendency to confabulate plausible but false future states. By injecting carefully crafted latent representations or conditioning inputs, attackers cause the agent to plan and act based on a convincingly predicted but entirely fabricated scenario. This is particularly dangerous in model-based reinforcement learning where agents use learned world models for planning, as the agent cannot distinguish between a genuine simulation and an adversarial hallucination.

The corruption of a saved simulation state such that when training or testing resumes from that checkpoint, the agent learns a malicious or compromised policy. Attackers modify:

• Agent parameters: Weights, biases, optimizer state
• Environment state: Object positions, velocities, episode counters
• Replay buffers: Stored transitions used for off-policy learning

Because checkpoints are often shared across teams and stored with less security scrutiny than production artifacts, they represent a high-value attack surface for supply chain compromises.