Spatial Anchor

A spatial anchor creates a persistent coordinate frame that is rigidly attached to a specific location in the physical world. This is achieved by storing a rich feature descriptor of the local visual environment (texture, edges, planar surfaces). When an application re-enters the space, the device's sensors scan the area, match the current visual features against the stored descriptor, and recalculate the device's 6DoF pose relative to the anchor. This allows virtual objects to appear world-locked, maintaining their position even if the user leaves and returns hours or days later.

• Key Mechanism: Visual-inertial odometry (VIO) combined with persistent local feature maps.
• Example: A virtual maintenance manual anchored to a specific machine on a factory floor remains fixed to that machine for all technicians across shifts.

For persistence beyond a single device or session, anchors are hosted in a cloud anchor service (e.g., Azure Spatial Anchors, ARCore Cloud Anchors). The service stores the anchor's feature descriptor and computed pose. Any device with the correct application and permissions can then locate the cloud anchor by scanning the environment. This enables multi-user collaborative experiences and cross-device persistence.

• Core Process: Anchor creation → feature extraction → upload to cloud service → sharing via identifier → remote device query and localization.
• Use Case: Multiple architects in different locations using HoloLens devices to view and interact with the same anchored 3D building model on a physical site plan.

The accuracy of a spatial anchor is paramount for believable AR. Precision is achieved through:

• Dense Feature Matching: Comparing hundreds of environmental features.
• Sensor Fusion: Combining visual data from cameras with inertial data from an IMU to correct for drift and fast motion.
• Refinement Over Time: Some systems continuously refine an anchor's pose as more observational data is collected, improving its stability.

This results in sub-centimeter to centimeter-level accuracy, allowing for precise alignment of virtual and physical objects, such as placing a virtual bolt into a real hole.

Spatial anchors must function in dynamic, real-world conditions. Robustness is engineered through:

• Invariant Feature Descriptors: Using features that are resistant to changes in lighting, seasonal decor, or minor object movement.
• Wide Baseline Relocalization: The ability to recognize an anchor from significantly different viewpoints.
• Handling of Occlusion: Temporary obstructions (like a person walking by) should not permanently break the anchor's localization.

Systems are tested for robustness against gradual lighting changes, non-structural scene modifications (e.g., moved chairs), and moderate geometry changes.

Anchors do not exist in isolation; they integrate with a device's broader spatial understanding pipeline. An anchor provides a stable root node for a local coordinate system. The device's real-time spatial mapping or world mesh generation can then be semantically linked to this anchor.

• Spatial Relationship: Enables queries like "place the virtual object 2 meters north of Anchor A and on the detected floor plane."
• Occlusion & Physics: The runtime environment mesh can be used to make virtual content occlude correctly behind real geometry anchored nearby.
• Navigation: Anchors can serve as key waypoints for pathfinding within a mapped environment.

In a session using multiple anchors or during extended use, the device maintains a local pose graph. Each anchor and keyframe from the device's tracking becomes a node, with edges representing measured spatial constraints. This graph allows for:

• Relative Localization: Understanding the position of all anchors relative to each other.
• Drift Distribution: When loop closure occurs (e.g., relocalizing to a previously seen anchor), the accumulated drift error can be distributed across the entire pose graph, keeping the virtual scene coherent.
• Multi-Anchor Scenes: Supporting large-scale experiences where content is pinned to many different locations in a building.

The foundational real-time process that enables spatial anchors. SLAM algorithms allow a device to build a map of an unknown environment while simultaneously tracking its own position within it. This creates the initial spatial understanding upon which persistent anchors are placed.

• Core Function: Creates the live, local coordinate system.
• Relation to Anchors: A spatial anchor is a persistent, globally consistent point registered within a SLAM-generated map.

The precise position and orientation that a spatial anchor defines and recalls. 6DoF Pose refers to the six degrees of freedom: three for translation (X, Y, Z) and three for rotation (roll, pitch, yaw).

• Anchor Output: When an application recalls an anchor, it retrieves the exact 6DoF pose of the virtual content relative to the world.
• Tracking Requirement: Continuous 6DoF device tracking (via VIO or Visual SLAM) is necessary to render content locked to an anchor.

The primary sensor fusion technique that provides the robust, real-time tracking needed to interact with spatial anchors. VIO combines camera images with inertial measurement unit (IMU) data to estimate device motion.

• Key Benefit: Maintains pose estimation during quick motion or temporary visual occlusion (e.g., a hand blocking the camera).
• System Dependency: Platforms like ARKit and ARCore use VIO as their core tracking engine, which anchors rely upon.

The critical mechanism that enables long-term persistence of spatial anchors by correcting accumulated drift. Loop closure occurs when a SLAM system recognizes it has returned to a previously mapped area.

• Problem it Solves: Odometry (VIO) inherently accumulates small errors in pose estimation over time, causing drift.
• Anchor Persistence: By recognizing a location and performing a loop closure adjustment, the system can correct the global map, ensuring an anchor's stored pose remains accurate to the physical world.

The dense geometric representation of the environment that provides context for anchor placement and interaction. Spatial mapping generates a world mesh—a 3D polygonal model of surfaces.

• Use Case for Anchors: Applications use the mesh for physics (e.g., placing a virtual ball on a real table), occlusion (virtual object behind a real couch), and intuitive placement.
• Semantic Understanding: Advanced systems may attach semantic labels (e.g., 'floor', 'wall') to mesh regions, allowing for rules like "place anchor only on horizontal planes."

The two primary persistence modalities, defining how anchor data is stored and shared.

• Cloud Anchors (e.g., Azure Spatial Anchors, ARCore Cloud Anchors): Anchor pose is resolved against a shared cloud-based spatial map. Enables multi-user, cross-device experiences where users see the same virtual content in the same place.
• Local Anchors: Anchor data is stored locally on the device. Useful for single-user, session-persistent experiences but cannot be shared reliably across different devices or after the local map is discarded.