keypoint detection

Semantic keypoints

Locating a predefined set of meaningful points on objects of a known category, such as the 17 body joints annotated in COCOCOCO, prop. n.: A widely used collection of everyday photos in which people outlined and named the objects, described each picture in sentences and marked body joints; it is used to train and compare vision systems., facial landmarks, hand joints or the corners of a product box; it is the basis of pose estimationpose estimation, n.: Working out the position and orientation of something from images: where a person's joints are and how their body is posed, or exactly how an object or camera is placed and turned in space.. Modern networks usually predict one heatmap per keypoint type, whose peak gives the location, or regress coordinates directly. Multi-person systems either detect each person and then find their keypoints (top-down) or detect all keypoints and group them into people (bottom-up). Accuracy is measured with object keypoint similarity (OKS), a scale-normalized distance used in place of intersection over unionintersection over union, n.: A score from 0 to 1 for how well two shapes overlap, such as a predicted box and the true box around an object: the shared area divided by the combined area. to compute keypoint APmean average precision, n.: The standard score for object detectors: it rewards finding all the objects while raising few false alarms, averaged over every type of object. Higher is better, up to 100., or with the percentage of correct keypoints (PCK).

Keypoint detection on the forklift's mast lets us estimate fork height from a single camera.

Interest points

Finding distinctive, repeatable image locations, such as corners and blobs, that can be found again in other views of the same scene. Classical detectors include Hans Moravec's interest operator (1977), the Harris corner detector (1988), FAST and the difference-of-Gaussians detector of SIFTscale-invariant feature transform, n.: A classic method for finding distinctive spots in a photo, such as corners and blobs, and describing them so the same spots can be matched in other photos taken from different distances or angles.; learned detectors such as SuperPoint (2018) followed. Matched interest points feed panorama stitching, visual odometryodometry, n.: Estimating how far and in which direction a robot or vehicle has moved by adding up its own motion, such as wheel turns or how camera images shift, like counting your steps., simultaneous localization and mappingsimultaneous localization and mapping, n.: How a robot, drone or headset builds a map of an unfamiliar place while at the same time working out where it is on that map, using its cameras or laser scanners. and structure from motionstructure from motion, n.: Recovering both the 3D shape of a scene and the spot from which each photo was taken, using nothing but a collection of overlapping photographs..