3D reconstruction
In plain wordsBuilding a 3D model of an object or place, its shape and often its colors, from photos, video or depth-sensor scans.
The recovery of the three-dimensional shape, and often the appearance, of objects or scenes from two-dimensional images or other measurements. The classical image-based pipeline is structure from motionstructure from motion, n.: Recovering both the 3D shape of a scene and the spot from which each photo was taken, using nothing but a collection of overlapping photographs., which estimates camera poses and a sparse point cloudpoint cloud, n.: A 3D picture made of many dots floating in space, each marking a spot on a surface, as captured by a laser scanner or worked out from photos., followed by multi-view stereo, which computes dense depth, and surface reconstruction, which turns points into a textured mesh. Active methods measure geometry directly with laser scanners, structured light, time-of-flight cameras or lidarlidar, n.: A sensor that measures distances by firing laser pulses and timing their reflections, building a 3D picture of its surroundings out of millions of points; widely used on robots and self-driving cars.. Neural methods represent the scene implicitly: signed-distance and occupancy networks (2019), neural radiance fieldsneural radiance field, n.: A way of capturing a real scene from a set of photos as a small neural network that can then show the scene from any new viewpoint, as if moving a camera through it. (2020) and Gaussian splattingGaussian splatting, n.: A way of turning photos of a real place into a 3D scene made of millions of tiny, soft, colored blobs that can be viewed from any angle in real time, even on a phone or headset. (2023) fit a scene representation to posed images by differentiable rendering. Since late 2023, feed-forward networks predict geometry directly from a handful of uncalibrated images: DUSt3R regresses 'pointmaps', and VGGT (2025) outputs camera parameters, depth maps and point tracks in under a second; other models reconstruct objects and people from a single image. Applications include mapping and surveying, cultural-heritage preservation, film and game assets, augmented and virtual reality, robotics, medicine and digital twinsdigital twin, n.: A detailed virtual copy of a real machine, factory, robot or city, kept in sync with live sensor data and used to monitor it, test changes safely and predict problems..
The drone's 300 photos give a 3D reconstruction of the bridge accurate enough to measure crack widths.
Origin
Measuring 3D shape from photographs began with 19th-century photogrammetry (Aimé Laussedat in France; Albrecht Meydenbauer, who named it in German in 1867). In computer vision, Lawrence Roberts's 1963 thesis recovered 3D blocks from images, and Christopher Longuet-Higgins's eight-point algorithm (1981) formalized the geometry of two views.
Usage note
'Reconstruction' implies recovering actual geometry from measurements; generating plausible 3D content from text or a single image is closer to generative modeling, though the two increasingly overlap.
Related
- structure from motionstructure from motion, n.: Recovering both the 3D shape of a scene and the spot from which each photo was taken, using nothing but a collection of overlapping photographs.
- point cloudpoint cloud, n.: A 3D picture made of many dots floating in space, each marking a spot on a surface, as captured by a laser scanner or worked out from photos.
- neural radiance fieldneural radiance field, n.: A way of capturing a real scene from a set of photos as a small neural network that can then show the scene from any new viewpoint, as if moving a camera through it.
- Gaussian splattingGaussian splatting, n.: A way of turning photos of a real place into a 3D scene made of millions of tiny, soft, colored blobs that can be viewed from any angle in real time, even on a phone or headset.
- depth estimationdepth estimation, n.: Working out how far away each point in a scene is from the camera, producing a 'depth map' that turns a flat picture into a sense of near and far.
- simultaneous localization and mappingsimultaneous localization and mapping, n.: How a robot, drone or headset builds a map of an unfamiliar place while at the same time working out where it is on that map, using its cameras or laser scanners.
- digital twindigital twin, n.: A detailed virtual copy of a real machine, factory, robot or city, kept in sync with live sensor data and used to monitor it, test changes safely and predict problems.
Sources
- Wang, S., Leroy, V., Cabon, Y., Chidlovskii, B. & Revaud, J. (2024). DUSt3R: Geometric 3D Vision Made Easy. CVPR 2024. arxiv.org/abs/2312.14132
- Wang, J., Chen, M., Karaev, N., Vedaldi, A., Rupprecht, C. & Novotny, D. (2025). VGGT: Visual Geometry Grounded Transformer. CVPR 2025. arxiv.org/abs/2503.11651
- Hartley, R. & Zisserman, A. (2004). Multiple View Geometry in Computer Vision, 2nd ed. Cambridge University Press.
Mentioned in
computer visioncomputer vision, n.: The part of AI that lets computers make sense of images and video: recognizing faces, reading signs, spotting tumors on scans or judging how far away a car is., calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time., depth estimationdepth estimation, n.: Working out how far away each point in a scene is from the camera, producing a 'depth map' that turns a flat picture into a sense of near and far., Gaussian splattingGaussian splatting, n.: A way of turning photos of a real place into a 3D scene made of millions of tiny, soft, colored blobs that can be viewed from any angle in real time, even on a phone or headset., lidarlidar, n.: A sensor that measures distances by firing laser pulses and timing their reflections, building a 3D picture of its surroundings out of millions of points; widely used on robots and self-driving cars., neural radiance fieldneural radiance field, n.: A way of capturing a real scene from a set of photos as a small neural network that can then show the scene from any new viewpoint, as if moving a camera through it., point cloudpoint cloud, n.: A 3D picture made of many dots floating in space, each marking a spot on a surface, as captured by a laser scanner or worked out from photos., occupancy gridoccupancy grid, n.: A robot's map of the world drawn as a grid of small squares or cubes, each marked with the chance that something solid is there, so the robot knows where it can safely go., scale-invariant feature transformscale-invariant feature transform, n.: A classic method for finding distinctive spots in a photo, such as corners and blobs, and describing them so the same spots can be matched in other photos taken from different distances or angles., score distillation samplingscore distillation sampling, n.: A trick for creating 3D objects from a text description by repeatedly asking a 2D image generator whether pictures of the object, seen from many angles, look right — and adjusting the object until they do., structure from motionstructure from motion, n.: Recovering both the 3D shape of a scene and the spot from which each photo was taken, using nothing but a collection of overlapping photographs.