Summary
The AI Computer Vision Research website showcases a new AI model called Segment Anything Model (SAM), which can accurately “cut out” any object in an image with a single click. This promptable segmentation system requires no additional training and has zero-shot generalization to unfamiliar objects and images. SAM’s extensible outputs can be used for a variety of applications, including image editing, 3D modeling, video tracking, and creative tasks like collaging.
Main Benefits
- Accurately segments any object in an image with a single click
- Requires no additional training and can generalize to unfamiliar objects and images
- Output masks can be used for a variety of applications, including image editing, 3D modeling, video tracking, and creative tasks like collaging
- Lightweight model design can run efficiently on CPU or GPU across a variety of platforms that support ONNX runtime
Possible Use Cases
– Image editing:
Use SAM’s output masks to make precise edits to specific objects in an image.
– 3D modeling:
Lift SAM’s output masks to 3D models for use in virtual reality or augmented reality applications.
– Video tracking:
Use SAM’s object masks to track objects across frames in videos.
– Creative tasks:
Use SAM’s output masks to create unique collages or other creative projects.
AI Edge
The power of AI is leveraged in the design and training of SAM. The model was trained on millions of images and masks collected through the use of a model-in-the-loop “data engine.” Researchers used SAM and its data to interactively annotate images and update the model. This cycle was repeated many times over to improve both the model and the dataset. The result is a lightweight model design that runs efficiently on CPU or GPU across a variety of platforms that support ONNX runtime.
Screenshots
