Back to Ultralytics

Inference API Testing

docs/en/platform/deploy/inference.md

8.4.12118.9 KB
Original Source

Inference

Ultralytics Platform provides browser-based inference for testing trained models and dedicated endpoints for programmatic access.

<!-- screenshot -->

Predict Tab

Every model with weights includes a Predict tab for browser-based inference:

  1. Navigate to your model
  2. Click the Predict tab
  3. Upload an image, use an example, or open your webcam
  4. Review the task-specific overlay, prediction summary, timing, and raw response

Models without weights show an empty state instead — train the model or upload weights first.

<!-- screenshot -->

Input Methods

The predict panel supports multiple input methods:

MethodDescription
Image uploadDrag and drop or click to upload an image
Example imagesClick built-in examples (dataset images or defaults)
Webcam captureLive camera feed with single-frame capture
mermaid
graph LR
    A[Upload Image]:::start --> D[Auto-Inference]:::proc
    B[Example Image]:::start --> D
    C[Webcam Capture]:::start --> D
    D --> E[Results + Overlays]:::out

    classDef start fill:#4CAF50,color:#fff
    classDef proc fill:#2196F3,color:#fff
    classDef out fill:#9C27B0,color:#fff

Upload Image

Drag and drop or click to upload:

  • Supported formats: JPEG, PNG, WebP, AVIF, HEIC, JP2, TIFF, BMP
  • Max size: 10 MB
  • Auto-inference: Results appear automatically after upload

!!! info "Auto-Inference"

The predict panel runs inference automatically when you upload an image, select an example, or capture a webcam frame. No button click is needed.

!!! note "Client-Side Resize"

Before uploading, the panel resizes the image so its longest side matches the selected `Image Size`, and requests normalized coordinates. This keeps browser testing fast; requests you send yourself are not resized.

Example Images

The predict panel shows up to two example images from your model's linked dataset, preferring the val split, then test, then train. If no dataset is linked, default examples are used:

ImageContent
bus.jpgStreet scene with vehicles
zidane.jpgSports scene with people

For OBB models, aerial images of boats and an airport are shown instead.

!!! tip "Preloaded Images"

Example images are preloaded when the page loads, so clicking an example triggers near-instant inference with no download wait.

Webcam

Click the webcam card to start a live camera feed:

  1. Grant camera permission when prompted
  2. Click the video preview to capture a frame
  3. Inference runs automatically on the captured frame
  4. Click again to restart the webcam

View Results

Inference results display the output appropriate to the model task: boxes, masks, keypoints, oriented boxes, classification scores, semantic coverage, or a depth map. Object results use the dataset class colors when available. The panel also shows preprocess, inference, postprocess, and network timing.

<!-- screenshot --> The results panel shows:

FieldDescription
Results summaryPer-detection list, or the top 5 classes for classification and semantic models
Speed statsPreprocess, inference, postprocess, and network (ms)
VersionsUltralytics and PyTorch versions, plus depth range or mask size where applicable
JSON responseRaw API response in a code block, with base64 map data elided

Two controls sit over the preview once results are in: click the image to enlarge it with overlays intact, and use the download button to save an annotated JPEG of the current result.

Inference Parameters

Adjust inference behavior with the three sliders below the image:

<!-- screenshot -->

ParameterRangeDefaultDescription
Confidence0.01 – 1.0, steps of 0.010.25Minimum confidence threshold
IoU0.0 – 0.95, steps of 0.010.7NMS IoU threshold
Image Size32 – 1280, steps of 32640Input resize dimension

!!! note "Auto-Rerun"

Changing any parameter automatically re-runs inference on the current image with a 500ms debounce. No need to re-upload.

Confidence Threshold

Filter predictions by confidence:

  • Higher (0.5+): Fewer, more certain predictions
  • Lower (0.1-0.25): More predictions, some noise
  • Default (0.25): Balanced for most use cases

IoU Threshold

Control Non-Maximum Suppression:

  • Higher (0.7+): Allow more overlapping boxes
  • Lower (0.3-0.5): Suppress overlapping detections more aggressively
  • Default (0.7): Balanced NMS behavior for most use cases

Deployment Predict

Each running dedicated endpoint includes a Predict tab directly on its deployment card. This uses the deployment's own inference service rather than the shared predict service, letting you test your deployed endpoint from the browser.

Dedicated Endpoint API

The API Docs card in the model Predict tab contains example Python, JavaScript, and cURL requests, pre-filled with the confidence, IoU, and image size currently set on the sliders. The URL and key are placeholders until you deploy the model — a Deploy button next to the code tabs jumps to the model's Deploy tab. After deployment, the deployment card's Code tab fills in that endpoint's URL and, for workspace owners, its bound API key, ready to copy and run.

Authentication

Include your API key in requests:

bash
Authorization: Bearer YOUR_API_KEY

!!! warning "API Key Required"

To run inference from your own scripts, notebooks, or apps, include an API key. Generate one in [`Settings > API Keys`](../account/api-keys.md). A dedicated endpoint accepts only the single key it was created with; the shared model API accepts any active key in the workspace, and public models also accept anonymous requests.

Endpoint

Dedicated endpoints take requests on their own URL:

http
POST https://YOUR_DEPLOYMENT_URL.run.app/predict

Shared inference uses the Platform API with the model's full path:

http
POST https://platform.ultralytics.com/api/models/{owner}/{project}/{model}/predict

Both accept the same multipart/form-data body and return the same response shape. With the Python SDK, use client.models.predict(owner, project, model, body=...) for shared inference or client.deployments.predict(owner, deployment, body=...) for a dedicated deployment:

python
from ultralytics_platform import Platform

client = Platform()  # reads ULTRALYTICS_API_KEY
with open("image.jpg", "rb") as f:
    results = client.models.predict("acme-vision", "inspection", "v3", body={"file": f, "conf": 0.25})

Request

=== "Python"

```python
import requests

url = "https://YOUR_DEPLOYMENT_URL.run.app/predict"
headers = {"Authorization": "Bearer YOUR_API_KEY"}
data = {"conf": 0.25, "iou": 0.7, "imgsz": 640}

with open("image.jpg", "rb") as image_file:
    response = requests.post(url, headers=headers, files={"file": image_file}, data=data)
print(response.json())
```

=== "cURL"

```bash
curl -X POST \
  "https://YOUR_DEPLOYMENT_URL.run.app/predict" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -F "[email protected]" \
  -F "conf=0.25" \
  -F "iou=0.7" \
  -F "imgsz=640"
```

=== "JavaScript"

```javascript
const formData = new FormData();
formData.append("file", fileInput.files[0]);
formData.append("conf", "0.25");
formData.append("iou", "0.7");
formData.append("imgsz", "640");

const response = await fetch(
  "https://YOUR_DEPLOYMENT_URL.run.app/predict",
  {
    method: "POST",
    headers: { Authorization: "Bearer YOUR_API_KEY" },
    body: formData,
  }
);

const result = await response.json();
console.log(result);
```

<!-- screenshot -->

Request Parameters

{% include "macros/platform-inference-parameters.md" %}

Response

json
{
    "images": [
        {
            "shape": [1080, 1920],
            "results": [
                {
                    "class": 0,
                    "name": "person",
                    "confidence": 0.92,
                    "box": { "x1": 100, "y1": 50, "x2": 300, "y2": 400 }
                },
                {
                    "class": 2,
                    "name": "car",
                    "confidence": 0.87,
                    "box": { "x1": 400, "y1": 200, "x2": 600, "y2": 350 }
                }
            ],
            "speed": {
                "preprocess": 1.2,
                "inference": 12.5,
                "postprocess": 2.3
            }
        }
    ],
    "metadata": {
        "imageCount": 1,
        "functionTimeAlive": 1284.51,
        "functionTimeCall": 0.018,
        "task": "detect",
        "version": {
            "ultralytics": "8.x.x",
            "torch": "2.6.0",
            "torchvision": "0.21.0",
            "python": "3.13.0"
        }
    }
}

<!-- screenshot -->

Response Fields

FieldTypeDescription
imagesarrayList of processed images, one entry per video frame for videos
images[].shapearrayImage dimensions [height, width]
images[].resultsarrayList of detections
images[].results[].classintClass index (integer ID)
images[].results[].namestringClass name
images[].results[].confidencefloatDetection confidence (0-1)
images[].results[].boxobjectBounding box coordinates
images[].semantic_maskobjectPer-pixel class map (semantic models only)
images[].depthobjectPer-pixel depth map (depth models only)
images[].speedobjectProcessing times in milliseconds
metadataobjectImage count, service timings, task, and Ultralytics/PyTorch versions

Task-Specific Responses

Response format varies by task:

=== "Detection"

```json
{
  "class": 0,
  "name": "person",
  "confidence": 0.92,
  "box": {"x1": 100, "y1": 50, "x2": 300, "y2": 400}
}
```

=== "Segmentation"

```json
{
  "class": 0,
  "name": "person",
  "confidence": 0.92,
  "box": {"x1": 100, "y1": 50, "x2": 300, "y2": 400},
  "segments": {"x": [100, 150, ...], "y": [50, 60, ...]}
}
```

=== "Semantic"

```json
{
  "results": [
    {"class": 0, "name": "road", "pixel_ratio": 0.42},
    {"class": 1, "name": "building", "pixel_ratio": 0.23}
  ],
  "semantic_mask": {
    "shape": [1080, 1920],
    "encoding": "png",
    "data": "<base64 PNG>"
  }
}
```

[Semantic segmentation](../../tasks/semantic.md) returns per-class pixel coverage (`pixel_ratio`, the fraction of image pixels assigned to each class) instead of per-object boxes, alongside `semantic_mask`: a base64-encoded PNG whose pixel values are class indices. Unlike the depth map, the mask is returned at the original image resolution (matching `images[].shape`), so it aligns per-pixel without resizing.

=== "Depth"

```json
{
  "results": [],
  "depth": {
    "shape": [480, 640],
    "encoding": "png",
    "data": "<base64 grayscale PNG>",
    "min": 0.31,
    "max": 79.9,
    "bits": 8
  }
}
```

[Depth estimation](../../tasks/depth.md) returns a dense per-pixel map instead of per-object results: a base64-encoded grayscale PNG where `depth = pixel × max / divisor` and a pixel value of `0` means no depth. The optional `bits` request parameter selects the quantization — `8` (default, uint8 PNG, divisor 255), `12`, or `16` (uint16 PNG, divisor 65535). The map is returned at model inference resolution (`imgsz`), so resize it to the image dimensions if you need per-pixel alignment. Decode it with any image library:

```python
import base64
import io

import numpy as np
from PIL import Image

depth = response["images"][0]["depth"]
pixels = np.asarray(Image.open(io.BytesIO(base64.b64decode(depth["data"]))))
meters = pixels * depth["max"] / (255.0 if depth["bits"] == 8 else 65535.0)  # 0 = no depth
```

=== "Pose"

```json
{
  "class": 0,
  "name": "person",
  "confidence": 0.92,
  "box": {"x1": 100, "y1": 50, "x2": 300, "y2": 400},
  "keypoints": {
    "x": [200, ...],
    "y": [75, ...],
    "visible": [0.95, ...]
  }
}
```

The `visible` array contains per-keypoint confidence scores (0-1 floats), not COCO-style 0-2 visibility flags.

=== "Classification"

```json
{
  "results": [
    {"class": 0, "name": "cat", "confidence": 0.95},
    {"class": 1, "name": "dog", "confidence": 0.03}
  ]
}
```

[Classification](../../tasks/classify.md) returns the top 5 classes by confidence, without boxes.

=== "OBB"

```json
{
  "class": 0,
  "name": "ship",
  "confidence": 0.89,
  "box": {
    "x1": 105,
    "y1": 48,
    "x2": 295,
    "y2": 55,
    "x3": 290,
    "y3": 395,
    "x4": 110,
    "y4": 402
  }
}
```

Rate Limits

The shared model API is limited to 20 requests/minute for each API key, signed-in caller, or anonymous IP. When throttled, the API returns 429 with a Retry-After header. See the full rate-limit reference for all endpoint categories.

!!! tip "Need More Throughput?"

Requests sent directly to a [dedicated endpoint](endpoints.md) do not pass through the Platform API rate limiter. The endpoint still sheds load with `429` and a `Retry-After` header when it is temporarily at capacity. For high-volume local inference, see the [Predict mode guide](../../modes/predict.md).

Error Handling

Common error responses:

CodeMessageSolution
400Invalid imageCheck file format, or that the model has trained weights
401UnauthorizedVerify API key
404Model not foundCheck the owner, project, and model names
413Input too largeReduce the file size below the endpoint limit
429Rate limitedWait and retry, or send requests directly to a dedicated endpoint
500Server errorRetry request
503Service unavailablePredict service starting up or unreachable; wait briefly and retry

FAQ

Can I run inference on video?

Both inference methods accept video files:

  • Dedicated endpoints accept video files directly. Supported formats (up to 100 MB): ASF, AVI, GIF, M4V, MKV, MOV, MP4, MPEG, MPG, TS, WEBM, WMV. Each frame is processed individually and results are returned per frame. See dedicated endpoints for details.
  • Shared inference (POST /api/models/{owner}/{project}/{model}/predict) uses the same predict service and accepts the same video formats. The browser Predict tab only selects images, so use the API or a dedicated endpoint for video.

How do I get the annotated image?

In the Predict tab, the download button over the preview saves the current result as an annotated JPEG. The API itself returns JSON predictions. To visualize those:

  1. Use predictions to draw boxes locally
  2. Use Ultralytics plot() method:
python
from ultralytics import YOLO

model = YOLO("yolo26n.pt")
results = model("image.jpg")
results[0].save("annotated.jpg")

See the Predict mode documentation for the full results API and visualization options.

What's the maximum image size?

  • Predict tab limit: 10 MB
  • API limit: 100 MB for both shared inference and dedicated endpoints
  • Auto-resize in the Predict tab: Images are resized to the selected Image Size before upload

Large images are automatically resized in the browser while preserving aspect ratio. Requests you send yourself are not resized, so images above the limit are rejected with 413.

Can I run batch inference?

The current API processes one image per request. For batch:

  1. Send separate requests for each image
  2. Distribute requests across dedicated endpoints when appropriate
  3. Use local inference for large batches

!!! example "Batch Inference with Python"

```python
import concurrent.futures

import requests

url = "https://YOUR_DEPLOYMENT_URL.run.app/predict"
headers = {"Authorization": "Bearer YOUR_API_KEY"}
images = ["img1.jpg", "img2.jpg", "img3.jpg"]


def predict(image_path):
    with open(image_path, "rb") as f:
        return requests.post(url, headers=headers, files={"file": f}).json()


with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
    results = list(executor.map(predict, images))
```