128 lines
11 KiB
Markdown
128 lines
11 KiB
Markdown
---
|
||
title: 'Automatic annotation'
|
||
linkTitle: 'Automatic annotation'
|
||
weight: 16
|
||
description: 'Automatic annotation of tasks'
|
||
---
|
||
|
||
Automatic annotation in CVAT is a tool that you can use
|
||
to automatically pre-annotate your data with pre-trained models.
|
||
|
||
CVAT can use models from the following sources:
|
||
|
||
- [Pre-installed models](#models).
|
||
- Models integrated from [Hugging Face and Roboflow](#adding-models-from-hugging-face-and-roboflow).
|
||
- {{< ilink "/docs/manual/advanced/serverless-tutorial" "Self-hosted models deployed with Nuclio" >}}.
|
||
- {{< ilink "/docs/enterprise/segment-anything-2-tracker" "AI agent functions (SAM2 tracking)" >}}
|
||
for CVAT Online and Enterprise.
|
||
|
||
The following table describes the available options:
|
||
|
||
| | Self-hosted | Online |
|
||
| ------------------------------------------- | ---------------------- | ------------------------------------------------------ |
|
||
| **Price** | Free | See [Pricing](https://www.cvat.ai/pricing/cvat-online) |
|
||
| **Models** | You have to add models | You can use pre-installed models |
|
||
| **Hugging Face & Roboflow <br>integration** | Not supported | Supported |
|
||
| **AI Agent Functions** | Supported (Enterprise) | Supported (SAM2 tracking available) |
|
||
|
||
See:
|
||
|
||
- [Running Automatic annotation](#running-automatic-annotation)
|
||
- [Labels matching](#labels-matching)
|
||
- [Models](#models)
|
||
- [Adding models from Hugging Face and Roboflow](#adding-models-from-hugging-face-and-roboflow)
|
||
|
||
## Running Automatic annotation
|
||
|
||
To start automatic annotation, do the following:
|
||
|
||
1. On the top menu, click **Tasks**.
|
||
1. Find the task you want to annotate and click **Action** > **Automatic annotation**.
|
||
|
||
|
||
|
||
1. In the Automatic annotation dialog, from the drop-down list, select a [model](#models).
|
||
1. [Match the labels](#labels-matching) of the model and the task.
|
||
1. (Optional) In case you need the model to return masks as polygons, switch toggle **Return masks as polygons**.
|
||
1. (Optional) In case you need to remove all previous annotations, switch toggle **Clean old annotations**.
|
||
1. (Optional) You can specify a **Threshold** for the model.
|
||
If not provided, the default value from the model settings will be used.
|
||
|
||
|
||
|
||
1. Click **Annotate**.
|
||
|
||
CVAT will show the progress of annotation on the progress bar.
|
||
|
||
|
||
|
||
You can stop the automatic annotation at any moment by clicking cancel.
|
||
|
||
## Labels matching
|
||
|
||
Each model is trained on a dataset and supports only the dataset's labels.
|
||
|
||
For example:
|
||
|
||
- DL model has the label `car`.
|
||
- Your task (or project) has the label `vehicle`.
|
||
|
||
To annotate, you need to match these two labels to give
|
||
CVAT a hint that, in this case, `car` = `vehicle`.
|
||
|
||
If you have a label that is not on the list
|
||
of DL labels, you will not be able to
|
||
match them.
|
||
|
||
For this reason, supported DL models are suitable only
|
||
for certain labels.
|
||
|
||
To check the list of labels for each model, see [Models](#models)
|
||
papers and official documentation.
|
||
|
||
## Models
|
||
|
||
Automatic annotation uses pre-installed and added models.
|
||
|
||
{{% alert title="Note" color="primary" %}}
|
||
For self-hosted solutions,
|
||
you need to
|
||
{{< ilink "/docs/administration/advanced/installation_automatic_annotation" "install Automatic Annotation first" >}}
|
||
and {{< ilink "/docs/manual/advanced/models" "add models" >}}.
|
||
{{% /alert %}}
|
||
|
||
List of pre-installed models:
|
||
|
||
<!--lint disable maximum-line-length-->
|
||
|
||
| Model | Description |
|
||
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||
| Attributed face detection | Three OpenVINO models work together: <br><br><li> [Face Detection 0205](https://docs.openvino.ai/2022.3/omz_models_model_face_detection_0205.html): face detector based on MobileNetV2 as a backbone with a FCOS head for indoor and outdoor scenes shot by a front-facing camera. <li>[Emotions recognition retail 0003](https://docs.openvino.ai/2022.3/omz_models_model_emotions_recognition_retail_0003.html#emotions-recognition-retail-0003): fully convolutional network for recognition of five emotions (‘neutral’, ‘happy’, ‘sad’, ‘surprise’, ‘anger’). <li>[Age gender recognition retail 0013](https://docs.openvino.ai/2022.3/omz_models_model_age_gender_recognition_retail_0013.html): fully convolutional network for simultaneous Age/Gender recognition. The network can recognize the age of people in the \[18 - 75\] years old range; it is not applicable for children since their faces were not in the training set. |
|
||
| RetinaNet R101 | RetinaNet is a one-stage object detection model that utilizes a focal loss function to address class imbalance during training. Focal loss applies a modulating term to the cross entropy loss to focus learning on hard negative examples. RetinaNet is a single, unified network composed of a backbone network and two task-specific subnetworks. <br><br>For more information, see: <li>[Site: RetinaNET](https://paperswithcode.com/lib/detectron2/retinanet) |
|
||
| Text detection | Text detector based on PixelLink architecture with MobileNetV2, depth_multiplier=1.4 as a backbone for indoor/outdoor scenes. <br><br> For more information, see: <li>[Site: OpenVINO Text detection 004](https://docs.openvino.ai/2022.3/omz_models_model_text_detection_0004.html) |
|
||
| YOLO v3 | YOLO v3 is a family of object detection architectures and models pre-trained on the COCO dataset. <br><br> For more information, see: <li>[Site: YOLO v3](https://docs.openvino.ai/2022.3/omz_models_model_yolo_v3_tf.html) |
|
||
| YOLO v7 | YOLOv7 is an advanced object detection model that outperforms other detectors in terms of both speed and accuracy. It can process frames at a rate ranging from 5 to 160 frames per second (FPS) and achieves the highest accuracy with 56.8% average precision (AP) among real-time object detectors running at 30 FPS or higher on the V100 graphics processing unit (GPU). <br><br> For more information, see: <li>[GitHub: YOLO v7](https://github.com/WongKinYiu/yolov7) <li>[Paper: YOLO v7](https://arxiv.org/pdf/2207.02696.pdf) |
|
||
|
||
<!--lint enable maximum-line-length-->
|
||
|
||
## Adding models from Hugging Face and Roboflow
|
||
|
||
In case you did not find the model you need, you can add a model
|
||
of your choice from [Hugging Face](https://huggingface.co/)
|
||
or [Roboflow](https://roboflow.com/).
|
||
|
||
{{% alert title="Note" color="primary" %}}
|
||
You cannot add models from Hugging Face and Roboflow to self-hosted CVAT.
|
||
{{% /alert %}}
|
||
|
||
<!--lint disable maximum-line-length-->
|
||
|
||
For more information,
|
||
see [Streamline annotation by integrating Hugging Face and Roboflow models](https://www.cvat.ai/post/integrating-hugging-face-and-roboflow-models).
|
||
|
||
This video demonstrates the process:
|
||
|
||
<iframe width="560" height="315" src="https://www.youtube.com/embed/SbU3aB65W5s" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
|
||
|
||
<!--lint enable maximum-line-length-->
|