Non-Rigid Registration

The core mathematical object in non-rigid registration is a deformation field—a vector-valued function that maps every point in a source 3D space to its corresponding location in a target space. This field is typically modeled as a smooth, continuous function (e.g., using radial basis functions, B-splines, or a neural network) to ensure physically plausible transformations without tearing or folding artifacts. The smoothness is enforced via regularization terms in the optimization objective.

Many modern non-rigid registration algorithms do not require pre-established point-to-point correspondences. Instead, they optimize the deformation field by minimizing a photometric or geometric alignment loss directly between the source and target data. Common metrics include:

• Chamfer Distance: Measures the average closest-point distance between two point clouds.
• Iterative Closest Point (ICP): Dynamically assigns correspondences during optimization.
• Differentiable Rendering Loss: For image-based registration, a neural renderer compares synthesized and target images. This approach is more robust to noise and partial overlaps.

Non-rigid registration techniques specialize based on the expected type of motion:

• Articulated Motion Models: Represent objects as a kinematic chain of rigid parts (bones) connected by joints. The deformation is parameterized by joint angles, and a skinning weight network defines how each bone influences surrounding vertices. This is standard for human or robotic motion.
• Elastic/Free-Form Deformation (FFD) Models: Represent motion as a smooth, continuous warp of a volumetric space, suitable for organic shapes like organs, cloth, or fluids. These are often driven by a control grid or neural network that defines the deformation field.

A common strategy is to learn a mapping from observed, deformed states back to a single, fixed canonical space. All observations of a deforming object (e.g., a person in different poses) are registered to this canonical template. This decouples the learning of static appearance and shape from the time-varying deformation, simplifying the reconstruction. The canonical space acts as a neutral reference configuration, and a separate deformation field is learned to warp points from this space to any observed frame.

For sequential data (e.g., video), non-rigid registration enforces temporal coherence—the idea that motion between frames is smooth and continuous. This is implemented via:

• Temporal Smoothness Losses: Penalize large accelerations or jerky motion in the deformation field.
• Motion Priors: Incorporate physical or statistical constraints (e.g., periodicity for walking, rigidity for some object parts) to guide optimization, especially with sparse or noisy data. These priors are crucial for realistic 4D reconstruction and preventing degenerate solutions.

State-of-the-art non-rigid registration is implemented within a fully differentiable framework. This allows the deformation field parameters (whether network weights or spline coefficients) to be optimized via gradient descent. The pipeline is end-to-end differentiable, from the input data (point clouds, images) through the deformation application to the final loss calculation. This integration with deep learning enables joint optimization with other tasks like novel view synthesis in Dynamic NeRF or scene flow estimation.

A deformation field is a continuous vector field that defines a mapping from points in a canonical 3D space to their observed positions in a deformed space at a specific time. It is the mathematical engine behind non-rigid registration.

• Core Mechanism: It outputs a 3D displacement vector for every input coordinate, modeling smooth, elastic transformations.
• Key Application: Central to Deformable NeRF methods, where a static canonical NeRF is warped to match each observed frame.
• Representation: Often parameterized by a Multilayer Perceptron (MLP) that takes (x, y, z, t) as input and outputs (Δx, Δy, Δz).

Scene flow estimation is the task of calculating the 3D motion vector of every point in a scene between two timesteps. It provides the dense correspondence field that non-rigid registration aims to compute.

• Dense Output: Unlike optical flow (2D), scene flow estimates full 3D translation for each 3D point.
• Input Data: Typically requires depth data or multi-view imagery to resolve the 3D motion ambiguity.
• Relation to Registration: Non-rigid registration can be viewed as the iterative optimization of a scene flow field that aligns two point clouds or volumes.

An articulated motion model represents deformation as a kinematic chain of rigid parts connected by joints. It is a powerful motion prior that constrains non-rigid registration for objects like humans, animals, or robots.

• Parameterization: Motion is defined by joint angles and bone lengths, drastically reducing degrees of freedom compared to a free-form deformation field.
• Skinning: A skinning weight network predicts how much each joint influences a given 3D point's motion.
• Use Case: Essential for human performance capture, where registering scans of a moving person relies on an underlying skeletal model.

Canonical space mapping is a strategy where all observations of a deforming object are mapped back to a single, fixed reference configuration. This disentangles learning appearance from learning motion.

• Core Idea: Learn a shared, high-fidelity model of shape and texture in a static canonical space. A separate deformation field maps observed points back to this space for rendering.
• Benefit: Improves reconstruction quality by allowing the model to aggregate information across all timesteps in a common coordinate frame.
• Technical Challenge: Requires learning a bi-directional mapping (from canonical to observed and vice-versa) that is consistent over time.

Dynamic NeRF extends the Neural Radiance Field framework to model scenes with motion. It inherently solves a non-rigid registration problem by learning a continuous 4D (x,y,z,t) representation from multi-view video.

• Representation: The neural network takes spatial coordinates and time as input, outputting color and density.
• Primary Approaches: Includes Deformable NeRF (uses a deformation field) and Direct Temporal Modeling (time is an additional network input).
• Output: Enables dynamic view synthesis—rendering photorealistic novel views at arbitrary viewpoints and timestamps.

4D reconstruction is the overarching goal of creating a time-varying, dynamic 3D model of a scene. Non-rigid registration is a fundamental algorithmic component within this pipeline.

• Definition: The process of capturing both the 3D geometry and its evolution over time from a sequence of images or videos.
• Output: A 4D model (3D + time) that can be used for playback, analysis, or free-viewpoint video.
• Applications: Ranges from human performance capture for film/VFX to analyzing mechanical stress tests or biological processes.