Segment Anything Model
In plain wordsA family of AI models from Meta that trace the exact outline of whatever you point to in a photo or video, whether by clicking on it, drawing a box around it or, in newer versions, naming it.
A family of promptable segmentation foundation modelsfoundation model, n.: A very large AI model, built once at great cost from vast amounts of text, images or code gathered mostly from the internet, that many products then reuse as their starting point, from chatbots to coding tools. released openly by Meta. The original SAM (2023) has three parts: a heavy image encoder (a vision transformervision transformer, n.: A way of applying the transformer, the design behind chatbots, to pictures: the image is cut into small square tiles that the model reads like words in a sentence. pretrained as a masked autoencodermasked autoencoder, n.: A way to teach a vision model without labels: hide most of an image's tiles and train the model to fill in the missing parts, so that it learns what images are made of.) that runs once per image; a prompt encoder for points, boxes or rough masks; and a lightweight mask decoder that returns masks in tens of milliseconds, offering several candidates with confidence scores when a prompt is ambiguous (a click on a shirt could mean the shirt or the whole person). It was trained on SA-1B, about 1.1 billion masks on 11 million licensed images, gathered with a 'data engine' in which model-assisted annotation gave way to fully automatic mask generation. SAM 2 (2024) extended prompting to video with a streaming memory that propagates masks across frames, and was faster on images. SAM 3 (November 2025) added promptable concept segmentation: given a short noun phrase or example boxes, it finds, segments and tracks every matching instance. SAM models are widely used for annotation, image editing, medical and scientific imaging, and as components of larger vision systems.
We used SAM to turn annotators' clicks into masks, which cut labeling time per image by more than half.
Origin
Introduced by Alexander Kirillov, Eric Mintun, Nikhila Ravi and colleagues at Meta AI's FAIR lab in 'Segment Anything' (April 2023; ICCV 2023), with the model and dataset released openly. SAM 2 (Nikhila Ravi et al.) followed in July 2024, with its paper in August, and SAM 3 (Nicolas Carion et al.) in November 2025.
Usage note
SAM produces masks without class names; naming what was segmented requires pairing it with a detector or text-grounding model, or using SAM 3. SAM 3D (November 2025) is a pair of companion models that reconstruct 3D objects and human bodies from single images.
Related
- image segmentationimage segmentation, n.: Dividing a picture into regions by deciding, pixel by pixel, which object or kind of surface each tiny dot belongs to, like coloring inside the outline of everything.
- instance segmentationinstance segmentation, n.: Finding each separate object in an image and tracing its exact outline, so that two people standing together get two distinct shapes rather than one blob.
- foundation modelfoundation model, n.: A very large AI model, built once at great cost from vast amounts of text, images or code gathered mostly from the internet, that many products then reuse as their starting point, from chatbots to coding tools.
- vision transformervision transformer, n.: A way of applying the transformer, the design behind chatbots, to pictures: the image is cut into small square tiles that the model reads like words in a sentence.
- open-vocabulary detectionopen-vocabulary detection, n.: Object detection where you type what to look for, such as 'red umbrella' or 'forklift', and the system finds it, even if it was never trained on that exact category.
- annotationannotation, n.: Adding information to raw data by hand or with software — marking objects in photos, tagging names in sentences, transcribing speech or ranking chatbot answers — so machines can learn from it.
Sources
- Kirillov, A. et al. (2023). Segment Anything. ICCV 2023. arxiv.org/abs/2304.02643
- Ravi, N. et al. (2024). SAM 2: Segment Anything in Images and Videos. arXiv:2408.00714. arxiv.org/abs/2408.00714
- Carion, N. et al. (2025). SAM 3: Segment Anything with Concepts. arXiv:2511.16719. arxiv.org/abs/2511.16719
Mentioned in
Meta AIMeta AI, prop. n.: The artificial intelligence research group of Meta, the company behind Facebook and Instagram, known for the PyTorch software and the Llama models. It is also the name of Meta's chatbot., image segmentationimage segmentation, n.: Dividing a picture into regions by deciding, pixel by pixel, which object or kind of surface each tiny dot belongs to, like coloring inside the outline of everything., instance segmentationinstance segmentation, n.: Finding each separate object in an image and tracing its exact outline, so that two people standing together get two distinct shapes rather than one blob., object trackingobject tracking, n.: Following objects from frame to frame in a video and keeping track of which is which, so a system knows that the car it sees now is the same car it saw a second ago., ICCVICCV, prop. n.: A major international computer vision conference, held in odd-numbered years, where much of the research on how machines interpret images and video is first presented., open-vocabulary detectionopen-vocabulary detection, n.: Object detection where you type what to look for, such as 'red umbrella' or 'forklift', and the system finds it, even if it was never trained on that exact category., sharpness-aware minimizationsharpness-aware minimization, n.: A training method that looks for settings where the error stays low even if the settings are nudged a little, rather than settings that are merely low at one precise point.