Sense-and-Avoid

The sensor suite provides raw environmental data. A robust system employs sensor fusion to combine complementary data streams from heterogeneous sensors:

• LiDAR provides high-resolution 3D point clouds for precise distance and shape measurement.
• Radar offers reliable velocity data and operates effectively in poor weather (rain, fog).
• Cameras deliver rich semantic context (e.g., object classification, traffic signs).
• Ultrasonic sensors are used for very short-range detection, crucial for low-speed maneuvers. The perception stack processes this fused data to perform object detection, tracking, and state estimation (position, velocity, acceleration), creating a dynamic model of the operational environment.

This software module identifies discrete entities from sensor data and predicts their future states. Detection algorithms (e.g., YOLO for vision, clustering for LiDAR) segment data into potential obstacles. Multi-object tracking (MOT) algorithms, such as the Kalman Filter or its non-linear variants like the Extended Kalman Filter (EKF), associate detections over time to form tracks. This provides a persistent estimate for each obstacle's trajectory, which is critical for calculating metrics like Time to Collision (TTC) and Closest Point of Approach (CPA). The output is a list of tracked dynamic and static obstacles with associated confidence levels.

This component evaluates the threat level posed by each tracked obstacle. It translates geometric relationships into probabilistic or deterministic risk metrics. Core calculations include:

• Time to Collision (TTC): Estimates the time until impact if current courses remain unchanged.
• Distance at Closest Point of Approach (DCPA) and Time to CPA (TCPA): Predict the minimum future separation.
• Collision Cone Analysis: Determines if the agent's current velocity vector lies within a cone of velocities that would cause a collision. The assessment often incorporates safety margins to account for sensor noise, actuation lag, and other uncertainties. The result is a prioritized list of collision threats.

Based on the risk assessment, this module selects or generates a safe evasive action. Approaches range from simple rule-based systems to complex optimization:

• Rule-Based: Follows predefined hierarchies (e.g., brake, then steer).
• Optimization-Based: Uses algorithms like Model Predictive Control (MPC) to solve for the optimal sequence of control inputs that avoids collisions while adhering to vehicle dynamics and comfort constraints.
• Velocity-Based: Employs Reciprocal Velocity Obstacle (RVO) or Optimal Reciprocal Collision Avoidance (ORCA) algorithms for cooperative, decentralized avoidance in multi-agent settings. The planner must respect the agent's kinodynamic constraints (e.g., maximum acceleration, turning radius) to ensure the generated maneuver is physically executable.

This is the physical interface that executes the planned maneuver. The low-level controller translates high-level commands (e.g., "turn left at 2 m/s²") into precise actuator signals for steering, throttle, and braking. Performance depends on:

• Control Latency: The delay from command generation to physical response.
• Actuator Dynamics: The physical limits of the steering motor or brake hydraulics. For aerial systems, this involves controlling rotors or control surfaces. In ground vehicles, it interfaces with the Electronic Stability Control (ESC) and Electric Power Steering (EPS) systems. The fidelity and speed of this component directly determine the system's ability to follow the planned avoidance trajectory.

A critical supervisory layer that provides a formal safety guarantee. It operates in parallel with the primary sense-and-avoid pipeline. The safety monitor continuously verifies that the planned or executed actions will not violate a predefined safety envelope (e.g., a minimum distance to all obstacles). If a violation is predicted, the monitor can trigger a Runtime Assurance (RTA) protocol, such as an Emergency Stop or a switch to a verified, simpler safety controller. This architecture, sometimes called a simplex or guardian system, is essential for certifying systems that use complex, non-verifiable algorithms (e.g., neural networks) in their primary planning stack.

Urban and suburban drone delivery services rely on sense-and-avoid to navigate around buildings, power lines, and other air traffic. Systems use a fusion of visual cameras, ultrasonic sensors, and radar to detect static and dynamic obstacles. Key challenges include:

• Low-altitude navigation amidst urban clutter.
• Dynamic obstacle prediction for birds and other UAVs.
• Compliance with regulatory Beyond Visual Line of Sight (BVLOS) requirements, which mandate robust, automated avoidance capabilities.

Electric Vertical Take-Off and Landing (eVTOL) aircraft for urban air taxi services require certifiable sense-and-avoid systems for dense airspace. This application pushes the limits of technology with:

• Cooperative avoidance using Vehicle-to-Everything (V2X) communication to share intent.
• Non-cooperative sensing using radar and electro-optical systems to detect intruders without transponders.
• Trajectory prediction for other aircraft to enable proactive, fuel-efficient rerouting long before a critical Time to Collision (TTC).

Drones for crop monitoring, spraying, and mapping operate in open but complex environments. Their sense-and-avoid systems are tailored for:

• Terrain following to maintain optimal altitude over uneven fields.
• Obstacle detection for isolated trees, poles, and irrigation equipment.
• GNSS-denied operation, where visual odometry and obstacle inflation techniques ensure safety when satellite signals are unreliable.

Rovers like NASA's Perseverance use autonomous navigation to drive between waypoints set by ground control. Their sense-and-avoid capability is characterized by:

• Extreme latency tolerance, as commands from Earth can take over 20 minutes.
• Stereo vision to build local occupancy grids and detect rocks, craters, and sand hazards.
• Conservative safety margins and low-speed operation due to the impossibility of physical repair, making robust obstacle detection paramount.

A Collision Avoidance System (CAS) is the overarching hardware and software architecture that implements sense-and-avoid. It integrates sensors, a perception stack, risk assessment algorithms, and an actuation controller. In aviation, a Traffic Alert and Collision Avoidance System (TCAS) is a specific, regulated type of CAS for aircraft. In automotive contexts, Automated Emergency Braking (AEB) is a CAS feature.

• Core Components: Sensor fusion, trajectory prediction, risk assessment, maneuver generation.
• Safety Certification: CAS designs often require formal verification (e.g., DO-178C for aviation, ISO 26262 for automotive) to prove deterministic behavior under all operational conditions.

Velocity Obstacle (VO) algorithms are a geometric foundation for decentralized collision avoidance. They calculate the set of an agent's velocities that would cause a future collision with an obstacle.

• Reciprocal Assumption: Optimal Reciprocal Collision Avoidance (ORCA) is a seminal algorithm that extends VO. It assumes all agents cooperate by sharing responsibility for avoidance, allowing each to efficiently compute a half-plane of 'safe' velocities.
• Application: Widely used in multi-robot systems and crowd simulation for its computational efficiency and guarantee of collision-free navigation.

These are the fundamental risk metrics that quantify the imminence of a collision, driving the sense-and-avoid decision loop.

• Time to Collision (TTC): Estimates the time until collision if two agents maintain their current relative velocity and heading. A primary trigger for alert levels.
• Closest Point of Approach (CPA): Comprises two values:
  • Distance to CPA (DCPA): The predicted minimum separation distance.
  • Time to CPA (TCPA): The time until that minimum separation occurs.
• Use Case: Maritime navigation systems (AIS) and aerospace rely heavily on CPA for long-range threat assessment.

Model Predictive Control (MPC) is an optimization-based strategy for generating avoidance maneuvers. It repeatedly solves a finite-horizon optimal control problem that explicitly includes collision constraints.

• Proactive Planning: Unlike reactive methods, MPC plans a short-term trajectory that optimizes for goal progress, passenger comfort, and energy efficiency while guaranteeing constraint satisfaction.
• Combination with CBFs: Often paired with Predictive Control Barrier Functions (CBFs) to provide formal, mathematical safety guarantees that the planned trajectory will keep the agent within a 'safe set'.

This is the perception core of the 'sense' phase. It involves algorithmically combining data from heterogeneous sensors (LiDAR, radar, cameras, ultrasonic) to create a unified, reliable representation of the environment.

• Redundancy and Reliability: Fusion mitigates the weaknesses of individual sensors (e.g., camera performance in low light, LiDAR in fog).
• Occupancy Grid Output: A common fused representation is a probabilistic Occupancy Grid, where each cell has a probability of being occupied, used for real-time collision checking.
• Key Challenge: Temporal alignment (synchronization) and spatial alignment (calibration) of sensor data streams.

Runtime Assurance (RTA) is a safety architecture for systems where the primary controller (e.g., a learning-based planner) cannot be fully verified. It wraps the primary controller with a verified safety monitor or safety controller.

• Operation: The RTA layer continuously checks the primary controller's commands. If a command is predicted to violate a safety property (e.g., cause a collision), it is overridden by a provably safe alternative.
• Certification Path: Enables the use of complex AI/ML components in safety-critical systems by providing a formal safety envelope. It directly addresses concerns about the predictability of neural networks in sense-and-avoid loops.