Skip to content
Last Dictionary
Plane
Vision, Speech & PerceptionDeep LearningHistory & Landmarks

R-CNN

proper nounAlso: Regions with CNN features, Fast R-CNN, Faster R-CNN, Mask R-CNN, region proposal network, RPN, RoI pooling, RoIAlign

In plain wordsA family of object detectors that first pick out image regions likely to contain objects, then use a neural network to label each region and tighten its box.

  1. 'Regions with CNN features': a two-stage approach to object detectionobject detection, n.: Finding every object of interest in a picture or video frame, saying what each one is and drawing a box around it, for example all the cars, cyclists and pedestrians in a street scene. and the family of models built on it. The original R-CNN took about 2,000 class-agnostic region proposals per image from the selective search algorithm, warped each to a fixed size, extracted features with a convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. pretrained on ImageNetImageNet, prop. n.: A huge collection of more than 14 million labeled photos, sorted into object categories, whose yearly recognition contest from 2010 to 2017 sparked the deep learning boom. and fine-tuned for detection, scored the regions with per-class support vector machinessupport vector machine, n.: A classic method that separates two groups of examples with the widest possible gap — like laying the broadest possible road between them. A mathematical shortcut lets it draw curved boundaries too. and refined them with bounding-box regression. It raised mean average precision on PASCAL VOC 2012 by more than 30% in relative terms, to 53.3%, but needed many seconds per image. Fast R-CNN (2015) computed convolutional features once per image and pooled them for each region (RoI pooling), training all parts jointly; Faster R-CNN (2015) replaced selective search with a learned region proposal network built on anchorsanchor box, n.: A ready-made box shape, placed at many spots across an image, that an object detector stretches and shifts to fit real objects instead of drawing boxes from scratch., making the detector a single network; Mask R-CNN (2017) added a branch predicting a pixel mask for each region and became the standard method for instance segmentationinstance segmentation, n.: Finding each separate object in an image and tracing its exact outline, so that two people standing together get two distinct shapes rather than one blob..

    For small, densely packed parts we still get better recall from Faster R-CNN than from the one-stage models.

Origin

Introduced by Ross Girshick, Jeff Donahue, Trevor Darrell and Jitendra Malik of UC Berkeley in a November 2013 arXiv paper published at CVPR 2014. Girshick's Fast R-CNN and Faster R-CNN (Shaoqing Ren, Kaiming He, Girshick, Jian Sun) followed at Microsoft Research, and Mask R-CNN (He, Georgia Gkioxari, Piotr Dollár, Girshick) at Facebook AI Research.

Usage note

Said letter by letter. 'Two-stage detector' is common shorthand for the R-CNN family, as opposed to one-stage detectors such as YOLOYOLO, prop. n.: A family of fast object-detection models that look at an image once and immediately output boxes and labels for everything they recognize, quick enough for live video.. Despite the 'CNN' in the name, region-based detectors can use transformer backbonesbackbone, n.: The main, usually pretrained part of a vision model that turns an image into useful internal features; smaller task-specific parts are attached to it, like limbs to a spine..

Related

See also
  • object detectionobject detection, n.: Finding every object of interest in a picture or video frame, saying what each one is and drawing a box around it, for example all the cars, cyclists and pedestrians in a street scene.
  • YOLOYOLO, prop. n.: A family of fast object-detection models that look at an image once and immediately output boxes and labels for everything they recognize, quick enough for live video.
  • anchor boxanchor box, n.: A ready-made box shape, placed at many spots across an image, that an object detector stretches and shifts to fit real objects instead of drawing boxes from scratch.
  • instance segmentationinstance segmentation, n.: Finding each separate object in an image and tracing its exact outline, so that two people standing together get two distinct shapes rather than one blob.
  • convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
  • non-maximum suppressionnon-maximum suppression, n.: A clean-up step in object detection: when the system draws several overlapping boxes around the same object, it keeps the most confident one and throws away the near-duplicates.
  • DETRDETR, prop. n.: An object detector built on the transformer, the design behind modern chatbots, that outputs a short list of objects in one go instead of sorting through thousands of guessed boxes.
Contrast with
  • YOLOYOLO, prop. n.: A family of fast object-detection models that look at an image once and immediately output boxes and labels for everything they recognize, quick enough for live video.

Sources

  1. Girshick, R., Donahue, J., Darrell, T. & Malik, J. (2014). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. CVPR 2014. arxiv.org/abs/1311.2524
  2. Ren, S., He, K., Girshick, R. & Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. NeurIPS 2015.
  3. He, K., Gkioxari, G., Dollár, P. & Girshick, R. (2017). Mask R-CNN. ICCV 2017. arxiv.org/abs/1703.06870

Mentioned in

object detectionobject detection, n.: Finding every object of interest in a picture or video frame, saying what each one is and drawing a box around it, for example all the cars, cyclists and pedestrians in a street scene., instance segmentationinstance segmentation, n.: Finding each separate object in an image and tracing its exact outline, so that two people standing together get two distinct shapes rather than one blob., YOLOYOLO, prop. n.: A family of fast object-detection models that look at an image once and immediately output boxes and labels for everything they recognize, quick enough for live video., anchor boxanchor box, n.: A ready-made box shape, placed at many spots across an image, that an object detector stretches and shifts to fit real objects instead of drawing boxes from scratch., DETRDETR, prop. n.: An object detector built on the transformer, the design behind modern chatbots, that outputs a short list of objects in one go instead of sorting through thousands of guessed boxes.