point cloud
In plain wordsA 3D picture made of many dots floating in space, each marking a spot on a surface, as captured by a laser scanner or worked out from photos.
A representation of 3D shape that records only sampled locations on the visible surfaces of objects or scenes, each optionally tagged with measurements such as color, return intensity or surface orientation. Lidarlidar, n.: A sensor that measures distances by firing laser pulses and timing their reflections, building a 3D picture of its surroundings out of millions of points; widely used on robots and self-driving cars. sweeps, depth cameras and survey scanners capture them directly, while structure from motionstructure from motion, n.: Recovering both the 3D shape of a scene and the spot from which each photo was taken, using nothing but a collection of overlapping photographs. and multi-view stereo reconstruct them from photographs; one sweep of a driving lidar can contain over 100,000 points. They are unordered, irregularly spaced and of varying density, which defeats standard convolutionsconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location.. PointNet (2017) handled this by applying a shared network to every point and pooling with a symmetric function (max), so that the result does not depend on point order; other approaches convert points to voxels or bird's-eye-view grids, use sparse convolutions or apply point transformerstransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.. Typical tasks are 3D object detection for autonomous drivingautonomous vehicle, n.: A vehicle, usually a car, truck or shuttle, that can drive itself, sensing the road, deciding what to do and steering, braking and accelerating with no human driving., segmentation, registration (aligning scans, classically by the iterative closest point algorithm, 1992) and surface reconstruction. Common file formats include PLY, PCD and LAS/LAZ for aerial survey data.
The lidar point cloud thins out beyond 80 meters, so distant pedestrians get only a handful of points.
Origin
The term arose in 3D scanning and surveying, where laser scanners and coordinate-measuring machines produced large sets of sampled surface points, and was common in graphics and reverse engineering by the 1990s. The Point Cloud Library (2011) and Charles Qi and colleagues' PointNet (CVPR 2017) brought point clouds into robotics software and deep learning.
Usage note
A point cloud has no connectivity: turning it into a watertight surface (mesh) is a separate reconstruction step, and gaps appear wherever surfaces were hidden from the sensor.
Related
- lidarlidar, n.: A sensor that measures distances by firing laser pulses and timing their reflections, building a 3D picture of its surroundings out of millions of points; widely used on robots and self-driving cars.
- 3D reconstruction3D reconstruction, n.: Building a 3D model of an object or place, its shape and often its colors, from photos, video or depth-sensor scans.
- structure from motionstructure from motion, n.: Recovering both the 3D shape of a scene and the spot from which each photo was taken, using nothing but a collection of overlapping photographs.
- depth estimationdepth estimation, n.: Working out how far away each point in a scene is from the camera, producing a 'depth map' that turns a flat picture into a sense of near and far.
- autonomous vehicleautonomous vehicle, n.: A vehicle, usually a car, truck or shuttle, that can drive itself, sensing the road, deciding what to do and steering, braking and accelerating with no human driving.
- Gaussian splattingGaussian splatting, n.: A way of turning photos of a real place into a 3D scene made of millions of tiny, soft, colored blobs that can be viewed from any angle in real time, even on a phone or headset.
Sources
- Qi, C. R., Su, H., Mo, K. & Guibas, L. J. (2017). PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. CVPR 2017.
- Besl, P. J. & McKay, N. D. (1992). A Method for Registration of 3-D Shapes. IEEE TPAMI 14(2), 239–256.
Mentioned in
3D reconstruction3D reconstruction, n.: Building a 3D model of an object or place, its shape and often its colors, from photos, video or depth-sensor scans., depth estimationdepth estimation, n.: Working out how far away each point in a scene is from the camera, producing a 'depth map' that turns a flat picture into a sense of near and far., Gaussian splattingGaussian splatting, n.: A way of turning photos of a real place into a 3D scene made of millions of tiny, soft, colored blobs that can be viewed from any angle in real time, even on a phone or headset., structure from motionstructure from motion, n.: Recovering both the 3D shape of a scene and the spot from which each photo was taken, using nothing but a collection of overlapping photographs.