Annotation Editor

Ultralytics Platform includes a powerful annotation editor for labeling images with bounding boxes, polygons, keypoints, oriented boxes, and classifications. The editor supports manual drawing and SAM-powered smart annotation.

graph TB
    subgraph Manual["Manual Tools"]
        A[Box] & B[Polygon] & C[Keypoint] & D[OBB] & E[Classify]
    end
    subgraph AI["AI-Assisted"]
        F[SAM Smart]
    end
    Manual --> H[Save Labels]
    AI --> H

Supported Task Types

The annotation editor supports all 5 YOLO task types:

!!! tip "Multi-Task Annotations"

All 5 annotation types are stored together on each image. You can switch the dataset's active task type without losing existing annotations — they are preserved and reappear when you switch back.

Task Details

??? info "Object Detection"

**What it does:** Identifies objects and their locations with axis-aligned bounding boxes.

**Label format:** [`class_id center_x center_y width height`](../../datasets/detect/index.md#ultralytics-yolo-format) (all normalized 0-1)

**Example:** `0 0.5 0.5 0.2 0.3` — Class 0 centered at (50%, 50%) with 20% width and 30% height

**Use cases:** Inventory counting, traffic monitoring, wildlife detection, security systems

??? info "Instance Segmentation"

**What it does:** Creates pixel-precise masks for each object instance.

**Label format:** [`class_id x1 y1 x2 y2 x3 y3 ...`](../../datasets/segment/index.md#ultralytics-yolo-format) (polygon vertices, normalized 0-1)

**Example:** `0 0.1 0.1 0.9 0.1 0.9 0.9 0.1 0.9` — Quadrilateral mask

**Use cases:** Medical imaging, autonomous vehicles, photo editing, agricultural analysis

??? info "Pose Estimation"

**What it does:** Detects body keypoints for skeleton tracking.

**Label format:** [`class_id cx cy w h kx1 ky1 v1 kx2 ky2 v2 ...`](../../datasets/pose/index.md#ultralytics-yolo-format)

- Visibility flags: `0`=not labeled, `1`=labeled but occluded, `2`=labeled and visible

**Example:** `0 0.5 0.5 0.2 0.3 0.6 0.7 2 0.4 0.8 1` — Person with 2 keypoints

**Use cases:** Sports analysis, physical therapy, animation, gesture recognition

??? info "Oriented Bounding Box (OBB)"

**What it does:** Detects rotated objects with angle-aware bounding boxes.

**Label format:** [`class_id x1 y1 x2 y2 x3 y3 x4 y4`](../../datasets/obb/index.md#yolo-obb-format) (four corner points, normalized)

**Example:** `0 0.1 0.1 0.9 0.1 0.9 0.9 0.1 0.9` — Rotated rectangle

**Use cases:** Aerial imagery, document analysis, manufacturing inspection, ship detection

??? info "Image Classification"

**What it does:** Assigns a single label to the entire image.

**Label format:** [Folder-based](../../datasets/classify/index.md#dataset-structure-for-yolo-classification-tasks) — images organized by class name (`train/cats/`, `train/dogs/`)

**Use cases:** Content moderation, quality control, medical diagnosis, scene recognition

Getting Started

To annotate images:

1. Navigate to your dataset
2. Click on an image to open the fullscreen viewer
3. Click Edit to enter annotation mode
4. Select your annotation tool from the toolbar
5. Draw annotations on the image
6. Click Save when finished

graph LR
    A[Open Dataset] --> B[Click Image]
    B --> C[Click Edit]
    C --> D[Draw Annotations]
    D --> E[Save]
    E --> F[Next Image]
    F --> B

    style C fill:#2196F3,color:#fff
    style D fill:#FF9800,color:#fff
    style E fill:#4CAF50,color:#fff

Annotation Modes

The editor provides two annotation modes, selectable from the toolbar:

Manual Annotation Tools

Bounding Box (Detect)

Draw rectangular boxes around objects:

1. Enter edit mode and select Draw
2. Click and drag to draw a rectangle
3. Release to complete the box
4. Select a class from the dropdown

!!! tip "Resize and Move"

- Drag 8 corner/edge handles to resize
- Drag the center to move
- Press `Delete` or `Backspace` to remove selected annotation

Polygon (Segment)

Draw precise polygon masks:

1. Enter edit mode and select Draw
2. Click to add vertices, or hold Shift and move the mouse to freehand-draw dense points
3. Click the first vertex, or press Enter or Escape to close the polygon
4. Select a class from the dropdown

!!! tip "Edit Vertices"

- Drag individual vertices to adjust
- Drag the entire polygon to move
- Click on a vertex and press `Delete` to remove it

Keypoint (Pose)

Annotate poses using skeleton templates. Select a template from the toolbar, click once to place all keypoints, then drag individual keypoints to adjust positions.

1. Enter edit mode and select Draw
2. Choose a skeleton template from the template picker in the toolbar

3. Click on the image to place all keypoints at once
4. Drag individual keypoints to adjust their positions
5. Press Enter to confirm or Escape to cancel

Built-in Skeleton Templates

The editor includes 5 built-in templates:

Custom Skeleton Templates

Create custom templates for any pose structure:

1. Click the + button next to the template picker
2. Optionally, click Start from... to load keypoints and connections from an existing template as a starting point
3. Place keypoints on the canvas by clicking
4. Name each keypoint and customize colors
5. Connect keypoints by selecting two points (connections are drawn automatically as you place sequential keypoints)
6. Save the template for reuse across your dataset

Custom templates are saved to your account and available in all pose datasets.

!!! tip "Template Workflow"

The template system replaces manual sequential keypoint placement. Instead of clicking 17 times for a COCO skeleton, select the Person template and click once — all keypoints are placed in the correct skeleton layout. Then drag to adjust positions for the specific pose.

!!! info "Keypoint Visibility"

Each keypoint has a visibility flag: `0` = not labeled, `1` = labeled but occluded, `2` = labeled and visible. Occluded keypoints (behind other objects) should be marked with visibility `1` — the model learns to infer their position.

Oriented Bounding Box (OBB)

Draw rotated boxes for angled objects:

1. Enter edit mode and select Draw
2. Click and drag to draw an initial box
3. Use the rotation handle to adjust angle
4. Drag corner handles to resize
5. Select a class from the dropdown

Classification (Classify)

Assign image-level class labels:

1. Enter edit mode
2. A side panel appears with class selection buttons
3. Click on class buttons or press number keys 1-9

Smart Annotation

Smart annotation adds model-assisted annotation to the editor. In Smart mode, you can use Segment Anything Model (SAM) for click-based annotation or use pretrained Ultralytics YOLO models and your own fine-tuned YOLO models to add predictions as annotations. Smart annotation is available for detect, segment, and OBB tasks.

SAM Smart Annotation

With a SAM model selected:

1. Enter edit mode and select Smart or press S
2. Click on the object you want to annotate — SAM generates an initial mask in real-time
3. Refine the mask with additional clicks: click outside the current mask to add coverage, or click inside the current mask to subtract regions
4. Press Enter or Escape to save the annotation, or enable auto-apply for one-click workflows

graph LR
    A[Press S] --> B[Left-click Object]
    B --> C[SAM Generates Mask]
    C --> D{Auto-apply?}
    D -->|Yes| E[Mask Applied Automatically]
    D -->|No| F{Accurate?}
    F -->|Yes| G[Enter to Save]
    F -->|No| H[Add +/- Points]
    H --> C

    style A fill:#2196F3,color:#fff
    style C fill:#FF9800,color:#fff
    style E fill:#4CAF50,color:#fff
    style G fill:#4CAF50,color:#fff

!!! tip "SAM Tips"

- Start with a click on the object center
- Click again outside the mask to expand coverage
- Click inside the mask to subtract unwanted regions
- Enable **auto-apply** (`A`) for one-click annotation
- Hold `Shift` while auto-apply is on to place multiple points before the mask is applied
- Works best for distinct objects with clear edges
- Use a few refinement clicks for elongated or overlapping objects

SAM smart annotation can generate:

• Polygons for segmentation tasks
• Bounding boxes for detection tasks
• Oriented boxes for OBB tasks

!!! warning "SAM Task Support"

SAM smart annotation is only available for **detect**, **segment**, and **OBB** tasks. Classification and pose tasks require manual annotation.

Auto-Apply Mode

Auto-apply mode speeds up Smart annotation by automatically saving the SAM mask after each click — no need to press Enter. Toggle it with the auto-apply button in the toolbar or press A.

!!! tip "When to Use Auto-Apply"

Auto-apply is ideal for datasets with well-separated objects where a single click produces an accurate mask. For complex or overlapping objects, turn auto-apply off and use multiple positive/negative points to refine the mask before saving.

SAM Model Selection

When Smart mode is active, a model picker appears in the toolbar. Five SAM models are available — choose based on the speed vs. accuracy trade-off that suits your dataset:

Switching models while Smart mode is active re-initializes the predictor for the current image automatically.

YOLO Smart Annotation

With a YOLO model selected, Smart annotation can add predictions from pretrained Ultralytics models or your own fine-tuned models.

1. Enter edit mode and select Smart or press S
2. Select a YOLO model from the model picker in the toolbar (Official or My Models)
3. Click Predict
4. Review the added annotations and make any needed corrections

!!! tip "YOLO Model Notes"

- The model picker only lists models that match the current dataset task.
- Duplicate predictions are skipped when they overlap an existing annotation of the same class at IoU `0.7` or higher.

Class Sidebar

The annotation editor includes a collapsible class sidebar on the right side of the canvas. The sidebar provides:

• Search classes: Filter the class list by typing in the search field. Press Enter on an exact match to select it, or create a new class if no match exists.
• Create new class inline: Click Add class at the bottom of the list, type a name, and optionally pick a custom color. Press Enter to create.
• Edit class name inline: Hover over a class name and click the pencil icon to rename it.
• Color picker: Click the color swatch next to any class to change its color.
• Per-class annotation count: Each class row shows a superscript count of annotations.
• Expand/collapse: Click the chevron to expand a class and see individual annotations listed below it.
• Bidirectional hover highlighting: Hovering an annotation on the canvas highlights it in the sidebar, and vice versa. The sidebar auto-scrolls and auto-expands to the relevant class.
• Hide/show individual annotations: Click the eye icon on any annotation row to toggle its visibility on the canvas.
• Delete annotations: Click the trash icon on any annotation row to delete it.
• Keyboard shortcuts: Press 1-9 to quickly select the first 9 classes.

Context Menu

Right-click on selected annotations to open a context menu with:

Visibility Controls

The visibility dropdown (eye icon) lets you toggle display of individual elements:

Crosshair Cursor

In edit mode, a crosshair overlay tracks the cursor position and displays pixel coordinates on the canvas. This helps place annotations with precision. Toggle it via the visibility dropdown.

SAM Hover Preview

In Smart mode, SAM provides a real-time hover preview before you click any points. This preview is available for detect, segment, and OBB tasks. Once you add refinement clicks, the preview updates to reflect the current mask and the annotation type for the active task.

Polygon Vertex Editing

For segment annotations, you can edit polygon vertices after drawing:

• Move vertices: Drag any vertex handle to reposition it
• Delete vertices: Select a vertex and press Delete to remove it

Class Management

Creating Classes

Define annotation classes for your dataset in the Classes tab:

1. Navigate to the Classes tab
2. Use the input field at the bottom to type a class name
3. Click Add or press Enter
4. A color is assigned automatically from the Ultralytics palette

Add New Class During Annotation

You can create new classes directly while annotating without leaving the editor:

1. Draw an annotation on the image
2. In the class dropdown, click Add New Class
3. Enter the class name
4. Press Enter to create and assign

This allows for a seamless workflow where you can define classes as you encounter new object types in your data.

!!! tip "Unified Classes Table"

All classes across your dataset are managed in a unified table. Changes to class names or colors apply throughout the entire dataset automatically.

Editing Classes

• Rename: Click a class name in the table to edit it inline
• Change color: Click the color swatch to open the color picker
• Search: Use the search field to filter classes by name
• Sort: Click column headers to sort by name, label count, or image count

Class Colors

Each class is assigned a color from the Ultralytics palette. You can customize colors using the color picker on the Classes tab. Colors are consistent across the platform for easy recognition.

Keyboard Shortcuts

Efficient annotation with keyboard shortcuts:

=== "General"

| Shortcut                      | Action                       |
| ----------------------------- | ---------------------------- |
| `Cmd/Ctrl+S`                  | Save annotations             |
| `Cmd/Ctrl+Z`                  | Undo                         |
| `Cmd/Ctrl+Y`                  | Redo                         |
| `Escape`                      | Save / Deselect / Exit       |
| `Delete` / `Backspace`        | Delete selected annotation   |
| `1-9`                         | Select class 1-9             |
| `Cmd/Ctrl+Scroll`             | Zoom in/out                  |
| `Cmd/Ctrl++` or `Cmd/Ctrl+=`  | Zoom in                      |
| `Cmd/Ctrl+-`                  | Zoom out                     |
| `Cmd/Ctrl+0`                  | Reset to fit                 |
| `Space+Drag`                  | Pan canvas when zoomed       |
| `Shift+Click`                 | Multi-select annotations     |
| `Cmd/Ctrl+A`                  | Select all annotations       |

=== "Modes"

| Shortcut | Action                          |
| -------- | ------------------------------- |
| `V`      | Manual mode (draw)              |
| `S`      | Smart mode (SAM or YOLO model)  |

=== "Drawing"

| Shortcut                | Action                                                                                 |
| ----------------------- | -------------------------------------------------------------------------------------- |
| `Click+Drag`            | Draw bounding box (detect/OBB)                                                         |
| `Click`                 | Add polygon point (segment) / Place skeleton (pose) / Place SAM point (smart)          |
| `Shift (hold) + Move`   | Freehand draw — continuously adds polygon vertices as the mouse moves                  |
| `Click inside mask`     | Subtract region from SAM mask (negative point)                                         |
| `Click outside mask`    | Add to SAM mask (positive point)                                                       |
| `Shift (hold) + Click`  | Place multiple SAM points before auto-apply commits (Smart mode, auto-apply on)        |
| `A`                     | Toggle auto-apply (Smart mode)                                                         |
| `Enter`                 | Complete polygon / Confirm pose / Save SAM annotation                                  |
| `Escape`                | Cancel pose / Save SAM annotation / Deselect / Exit                                    |

=== "Arrange (Z-Order)"

| Shortcut           | Action         |
| ------------------ | -------------- |
| `Cmd/Ctrl+]`       | Bring forward  |
| `Cmd/Ctrl+[`       | Send backward  |
| `Cmd/Ctrl+Shift+]` | Bring to front |
| `Cmd/Ctrl+Shift+[` | Send to back   |

??? tip "View All Shortcuts"

Click the keyboard icon in the annotation toolbar to open the shortcuts reference.

Undo/Redo

The annotation editor maintains a full undo/redo history:

• Undo: Cmd/Ctrl+Z
• Redo: Cmd/Ctrl+Y

History tracks:

• Adding annotations (single and batch)
• Editing annotations (move, resize, rotate)
• Deleting annotations (single and batch)
• Changing classes (single and batch)
• Reordering annotations (z-order)
• Editing polygon vertices (add, remove, move)
• Moving keypoints

!!! info "Unlimited Undo"

The undo stack has no fixed limit — you can undo all changes made during the current editing session, back to the original state of the image when you clicked `Edit`.

Saving Annotations

Annotations are saved when you click Save or press Cmd/Ctrl+S:

• Save: Click the save button or press Cmd/Ctrl+S
• Cancel: Click cancel to discard changes
• Escape: Saves if there are unsaved changes, otherwise exits edit mode

!!! warning "Save Your Work"

Always save before navigating to another image. Unsaved changes will be lost.

FAQ

How accurate is SAM annotation?

SAM provides high-quality masks for most objects. Accuracy depends on:

• Object distinctiveness from background
• Image quality and resolution
• Number of positive/negative points provided

For best results, start with a click on the object center, then use outside-mask clicks to add coverage and inside-mask clicks to subtract nearby objects or background.

Can I import existing annotations?

Yes, upload your dataset with YOLO-format label files. The Platform automatically parses and displays them in the editor.

How do I annotate multiple objects of the same class?

After drawing an annotation:

1. Keep the same class selected
2. Draw the next annotation
3. Repeat until all objects are labeled

The keyboard shortcut 1-9 quickly selects classes.

Can I train on partially annotated datasets?

Yes, but for best results:

• Label all objects of your target classes in each image
• Use the label filter set to Unlabeled to identify images that still need annotation
• Unlabeled images are excluded from training; only labeled images contribute to the loss

Which SAM model should I use?

SAM 3 is the default and the latest generation model — start there for the highest quality masks. Switch to SAM 2.1 Small for a faster interactive workflow on common objects, or SAM 2.1 Large when you need higher mask precision on complex shapes. Use SAM 2.1 Tiny for maximum speed on simple, high-contrast objects.

Which tasks support SAM smart annotation?

SAM smart annotation is available for detect, segment, and OBB tasks. Classification and pose tasks use manual annotation only.

Can I create custom skeleton templates for pose annotation?

Yes. Click the + button next to the skeleton template picker to open the template editor. Place keypoints, name them, draw connections, and save. Custom templates are stored in your account and available across all pose datasets.

How do I switch between skeleton templates?

Click the template picker dropdown in the annotation toolbar. Select any built-in template (Person, Hand, Face, Dog, Box) or your saved custom templates. The selected template determines which keypoints are placed when you click on the image.