Photometric Stereo

The algorithm requires precise knowledge of the direction and intensity of each light source for every input image. The distant light assumption means the light source is effectively at infinity, so the illumination direction is constant across the entire object's surface. This simplifies the lighting model to a directional vector, avoiding complex calculations for attenuation and positional variance. In practice, this is achieved using a light stage or calibrated laboratory setup.

The surface is assumed to be a perfect Lambertian reflector. This means it reflects light equally in all directions, and its brightness depends only on the angle between the surface normal and the light direction, not the viewing angle. The reflected radiance is modeled as: L = ρ * (n · l) * I, where ρ is the albedo, n is the surface normal, l is the light direction, and I is the light intensity. This linear relationship is crucial for solving the system of equations. Real-world materials with specular highlights or subsurface scattering violate this assumption and cause errors.

The camera is assumed to be in a fixed position viewing the object under all different lightings. Furthermore, an orthographic projection model is often used, which assumes all light rays entering the camera are parallel. This eliminates perspective distortion and simplifies the relationship between image coordinates and 3D surface points. In many implementations, a perspective camera with a long focal length (approximating orthography) is used, but the fixed viewpoint remains non-negotiable for the core linear solution.

The albedo (base color or diffuse reflectance) of the surface point must be constant across all input images for a given pixel. The classical algorithm solves for both the surface normal and a single albedo value per pixel. If the albedo is spatially varying but constant per point (e.g., a painted object), the algorithm can recover it. Problems arise with textureless regions (where the solution is ambiguous) or materials where albedo changes with viewing/lighting angle, which is non-Lambertian behavior.

The model assumes no shadows (cast or attached), no interreflections (light bouncing from one surface point to another), and no ambient light. Shadows create pixels where the light source is not visible, breaking the linear lighting equation. Interreflections add non-local lighting contributions, meaning a point's brightness depends on the geometry of other surfaces. Ambient light adds an unknown constant offset to pixel values. All these effects introduce non-linearities that the classical linear solution cannot handle, leading to corrupted normal and albedo estimates.

A minimum of three images under different, non-coplanar lighting directions are required to uniquely solve for the two degrees of freedom of the surface normal and the albedo at each pixel. With fewer lights, the system is under-constrained. If all light directions lie in a single plane (are coplanar), the component of the normal perpendicular to that plane becomes ambiguous. Using more than three lights (e.g., 10-50) creates an over-determined system that can be solved via least squares, improving robustness to noise and minor model violations.

A mathematical function that defines how light is reflected at an opaque surface. It describes the ratio of reflected radiance to incident irradiance as a function of the illumination direction and viewing direction. Key aspects:

• Physically Based: Models real-world light interaction.
• 4D Function: Takes incoming light direction (θᵢ, φᵢ) and outgoing view direction (θᵣ, φᵣ).
• Core of Shading: The BRDF is the fundamental model used in rendering engines to compute a surface's color under given lighting. Photometric stereo's goal is often to estimate a simplified or full BRDF for each point on a surface.

The overarching process of estimating the underlying 3D scene properties—such as geometry, materials (BRDF/SVBRDF), and lighting—from a set of 2D observations (images). It effectively inverts the traditional graphics rendering pipeline.

• Broader than Photometric Stereo: While photometric stereo typically solves for normals and albedo under known lights, inverse rendering often jointly optimizes for unknown geometry, complex materials, and unknown lighting.
• Uses Optimization: Heavily relies on differentiable rendering to compute gradients and update scene parameters.
• Modern Approaches: Often uses neural representations (like NeRFs) and deep learning to solve this ill-posed problem.

A texture map that encodes surface normal vectors as RGB colors (where R, G, B correspond to X, Y, Z components). It is used to simulate high-resolution surface detail (bumps, scratches, wrinkles) on a low-polygon 3D model without modifying its underlying geometry.

• Direct Output: The primary output of a classic photometric stereo algorithm is a normal map.
• Real-time Application: Normal maps are a staple in real-time rendering for games and VR, providing detailed lighting from simple geometry.
• From Normals to Bump: A normal map provides more accurate lighting information than a simpler height map (bump map) because it directly defines the surface orientation.

A controlled, spherical illumination system—typically a dome equipped with hundreds of programmable LED light sources and synchronized cameras—used to capture the complete reflectance field of an object or human face.

• Data Capture for Modeling: It is the premier hardware apparatus for capturing the data required for high-fidelity photometric stereo, SVBRDF estimation, and relightable avatars.
• Process: The subject remains static while lights are sequentially activated, capturing images under many known lighting directions from a fixed viewpoint—the ideal input for photometric stereo.
• Output: Enables photorealistic relighting and appearance cloning in visual effects.

The computer vision task of separating a single image of an object into its intrinsic components, primarily albedo (base color/texture) and shading (the effects of lighting and shape). More advanced decompositions can also output specular highlights and shadows.

• Related Problem: Photometric stereo can be seen as a multi-image, more constrained form of appearance decomposition where lighting is known.
• Challenges: Highly ill-posed from a single image (the "shape from shading" problem).
• Applications: Enables material editing, relighting, and consistency for augmented reality.

A computer graphics rendering methodology that aims to simulate the physical behavior of light and materials using measured surface properties and energy-conserving shading models. It provides a consistent, realistic appearance under any lighting environment.

• Uses BRDFs: PBR pipelines are built on microfacet BRDF models (like GGX).
• Material Inputs: Uses parameters like Base Color, Metallic, Roughness, and Normal maps—the types of properties photometric stereo and inverse rendering aim to recover.
• Industry Standard: The foundation for modern game engines (Unreal Engine, Unity) and offline renderers. Photometric stereo provides a way to create real-world PBR assets from photographs.