Simultaneous Localization and Mapping (SLAM)

The front-end is responsible for low-latency processing of raw sensor data to estimate incremental motion (odometry). It handles feature extraction (e.g., using SIFT or learned features), data association (matching features between frames), and initial motion estimation. Common techniques include Visual Odometry (VO) for cameras and Visual-Inertial Odometry (VIO) when fused with an IMU. This module provides a high-frequency, but drift-prone, pose estimate to the back-end.

The back-end performs global optimization to correct the accumulated drift from the front-end. It constructs a pose graph where nodes represent robot poses and edges represent constraints (from odometry or loop closures). When the robot recognizes a previously visited location (loop closure), it adds a new constraint to the graph. Algorithms like Bundle Adjustment or pose-graph optimization (using libraries like g2o or GTSAM) then solve for the most globally consistent set of poses and map points, minimizing reprojection error.

The map is the persistent model of the environment. Representations vary based on application:

• Sparse Feature Maps: Store only distinct 3D landmarks (points), efficient for localization.
• Dense Maps: Model every surface, often as a point cloud or mesh, crucial for navigation and manipulation.
• Occupancy Grids: Probabilistic 2D or 3D grids where each cell holds the probability of being occupied, ideal for path planning.
• Semantic Maps: Augment geometric maps with object labels (from Semantic Segmentation), enabling task-oriented reasoning.

SLAM systems rely on a suite of complementary sensors. Sensor fusion algorithms, like the Kalman Filter or Particle Filter, combine their data for robustness.

• Cameras (Monocular, Stereo, RGB-D): Provide rich visual texture and, in stereo/RGB-D, direct depth.
• LiDAR: Provides precise, long-range 3D point clouds but is textureless.
• Inertial Measurement Unit (IMU): Measures acceleration and angular rate, providing high-frequency motion data critical for stability during rapid movements or visual degradation.
• Wheel Encoders: Provide proprioceptive odometry.

This module is critical for correcting long-term drift. It answers the question: "Have I been here before?" It generates a compact descriptor (or "fingerprint") of the current view and performs Approximate Nearest Neighbor (ANN) search against a database of past descriptors. Techniques range from bag-of-words models on visual features to learned graph neural network embeddings. A successful match triggers a loop closure event, sending a constraint to the back-end optimizer.

Once a map is built, the system must continuously localize the robot within it. This tracking process is similar to the front-end but matches current sensor observations to the existing global map rather than just the previous frame. It is a simpler, more efficient problem than full SLAM. Robust localization often uses a particle filter (for Monte Carlo Localization) to maintain multiple pose hypotheses, especially in ambiguous or dynamic environments.

The process of estimating a robot's ego-motion (translation and rotation) by analyzing the geometric changes between consecutive camera frames. It is a core component of visual SLAM systems.

• Key Function: Provides a continuous, high-frequency estimate of camera movement.
• Challenge: Suffers from scale ambiguity in monocular systems and drift over time due to incremental error accumulation.
• Role in SLAM: Acts as the primary motion estimator, with loop closure and global optimization used to correct its accumulated drift.

A sensor fusion technique that tightly couples visual data from a camera with inertial data from an Inertial Measurement Unit (IMU) to estimate a platform's 6-degree-of-freedom pose and velocity.

• Advantage: The IMU provides metric scale, high-frequency motion estimates, and robustness during periods of poor visual tracking (e.g., fast motion, blur).
• Mechanism: Uses filters (like an Extended Kalman Filter) or optimization-based methods to fuse asynchronous sensor data.
• Outcome: Produces more accurate, robust, and scale-aware odometry than visual-only methods, forming the backbone of modern SLAM on mobile devices and drones.

A photogrammetric optimization technique that refines the 3D structure of a scene (landmark positions) and the camera poses by minimizing the total reprojection error—the difference between observed image points and projected 3D points.

• Purpose in SLAM: Serves as the global optimization backend. After loop closure is detected, bundle adjustment is run to correct drift and produce a globally consistent map and trajectory.
• Scale: Can be performed locally (windowed) for real-time operation or globally for final map refinement.
• Output: The optimal, non-linear least-squares solution for all estimated parameters.

The critical process of recognizing a previously visited location and using this constraint to correct the accumulated drift in the robot's estimated trajectory and map.

• Detection: Often uses visual place recognition (comparing current camera view to a database of past views) or lidar scan matching.
• Effect: Introduces a strong constraint in the pose graph, allowing Bundle Adjustment or pose graph optimization to distribute the correction error backward through the entire trajectory.
• Significance: Without loop closure, SLAM is just odometry; loop closure enables the creation of a globally consistent, usable map.

A probabilistic, discretized representation of an environment where each cell (voxel in 3D) stores the estimated probability that it is occupied by an obstacle.

• Use Case: Common in LiDAR SLAM and robotic navigation. It represents the map in a form directly usable for path planning and collision avoidance.
• Mapping: Built by integrating successive sensor readings (e.g., lidar scans) using inverse sensor models within a Bayesian framework (e.g., log-odds updates).
• Characteristic: A dense representation, in contrast to sparse feature-based maps, making it suitable for navigation in cluttered, unstructured environments.

A streamlined optimization framework used in SLAM where the map is represented as a graph. Nodes represent robot poses (positions and orientations), and edges represent constraints between poses derived from odometry or loop closure measurements.

• Efficiency: By marginalizing out the 3D landmark points, it optimizes only the robot's trajectory, which is far more computationally efficient than full Bundle Adjustment.
• Objective: Minimizes the error between the relative pose measurements (edges) and the predicted relative poses based on the node estimates.
• Application: The standard backend for large-scale, long-term SLAM systems where maintaining a full bundle adjustment problem is intractable.