Back to Ultralytics

Inference API Testing

docs/en/platform/deploy/inference.md

8.4.10515.4 KB
Original Source

Inference

Ultralytics Platform provides browser-based inference for testing trained models and dedicated endpoints for programmatic access.

<!-- screenshot -->

Predict Tab

Every model includes a Predict tab for browser-based inference:

  1. Navigate to your model
  2. Click the Predict tab
  3. Upload an image, use an example, or open your webcam
  4. Review the task-specific overlay, prediction summary, timing, and raw response

<!-- screenshot -->

Input Methods

The predict panel supports multiple input methods:

MethodDescription
Image uploadDrag and drop or click to upload an image
Example imagesClick built-in examples (dataset images or defaults)
Webcam captureLive camera feed with single-frame capture
mermaid
graph LR
    A[Upload Image]:::start --> D[Auto-Inference]:::proc
    B[Example Image]:::start --> D
    C[Webcam Capture]:::start --> D
    D --> E[Results + Overlays]:::out

    classDef start fill:#4CAF50,color:#fff
    classDef proc fill:#2196F3,color:#fff
    classDef out fill:#9C27B0,color:#fff

Upload Image

Drag and drop or click to upload:

  • Supported formats: JPEG, PNG, WebP, AVIF, HEIC, JP2, TIFF, BMP, DNG, MPO
  • Max size: 10MB
  • Auto-inference: Results appear automatically after upload

!!! info "Auto-Inference"

The predict panel runs inference automatically when you upload an image, select an example, or capture a webcam frame. No button click is needed.

Example Images

The predict panel shows example images from your model's linked dataset. If no dataset is linked, default examples are used:

ImageContent
bus.jpgStreet scene with vehicles
zidane.jpgSports scene with people

For OBB models, aerial images of boats and airports are shown instead.

!!! tip "Preloaded Images"

Example images are preloaded when the page loads, so clicking an example triggers near-instant inference with no download wait.

Webcam

Click the webcam card to start a live camera feed:

  1. Grant camera permission when prompted
  2. Click the video preview to capture a frame
  3. Inference runs automatically on the captured frame
  4. Click again to restart the webcam

View Results

Inference results display the output appropriate to the model task: boxes, masks, keypoints, oriented boxes, classification scores, semantic coverage, or a depth map. Object results use the dataset class colors when available. The panel also shows preprocess, inference, postprocess, and network timing.

<!-- screenshot --> The results panel shows:

FieldDescription
Results summaryDetections, classifications, or semantic class coverage
Speed statsPreprocess, inference, postprocess, and network (ms)
JSON responseRaw API response in a code block

Inference Parameters

Adjust inference behavior with the three sliders below the image:

<!-- screenshot -->

ParameterRangeDefaultDescription
Confidence0.01 – 1.0, steps of 0.010.25Minimum confidence threshold
IoU0.0 – 0.95, steps of 0.010.7NMS IoU threshold
Image Size32 – 1280, steps of 32640Input resize dimension

!!! note "Auto-Rerun"

Changing any parameter automatically re-runs inference on the current image with a 500ms debounce. No need to re-upload.

Confidence Threshold

Filter predictions by confidence:

  • Higher (0.5+): Fewer, more certain predictions
  • Lower (0.1-0.25): More predictions, some noise
  • Default (0.25): Balanced for most use cases

IoU Threshold

Control Non-Maximum Suppression:

  • Higher (0.7+): Allow more overlapping boxes
  • Lower (0.3-0.5): Suppress overlapping detections more aggressively
  • Default (0.7): Balanced NMS behavior for most use cases

Deployment Predict

Each running dedicated endpoint includes a Predict tab directly on its deployment card. This uses the deployment's own inference service rather than the shared predict service, letting you test your deployed endpoint from the browser.

Dedicated Endpoint API

The API Docs card in the model Predict tab contains example Python, JavaScript, and cURL requests. The examples use placeholders until you deploy the model. After deployment, the deployment card's Code tab fills in its endpoint URL and the API key available to your workspace.

Authentication

Include your API key in requests:

bash
Authorization: Bearer YOUR_API_KEY

!!! warning "API Key Required"

To run inference from your own scripts, notebooks, or apps, include an API key. Generate one in [`Settings > API Keys`](../account/api-keys.md).

Endpoint

http
POST https://YOUR_DEPLOYMENT_URL.run.app/predict

Request

=== "Python"

```python
import requests

url = "https://YOUR_DEPLOYMENT_URL.run.app/predict"
headers = {"Authorization": "Bearer YOUR_API_KEY"}
data = {"conf": 0.25, "iou": 0.7, "imgsz": 640}

with open("image.jpg", "rb") as image_file:
    response = requests.post(url, headers=headers, files={"file": image_file}, data=data)
print(response.json())
```

=== "cURL"

```bash
curl -X POST \
  "https://YOUR_DEPLOYMENT_URL.run.app/predict" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -F "[email protected]" \
  -F "conf=0.25" \
  -F "iou=0.7" \
  -F "imgsz=640"
```

=== "JavaScript"

```javascript
const formData = new FormData();
formData.append("file", fileInput.files[0]);
formData.append("conf", "0.25");
formData.append("iou", "0.7");
formData.append("imgsz", "640");

const response = await fetch(
  "https://YOUR_DEPLOYMENT_URL.run.app/predict",
  {
    method: "POST",
    headers: { Authorization: "Bearer YOUR_API_KEY" },
    body: formData,
  }
);

const result = await response.json();
console.log(result);
```

<!-- screenshot -->

Request Parameters

{% include "macros/platform-inference-parameters.md" %}

Response

json
{
    "images": [
        {
            "shape": [1080, 1920],
            "results": [
                {
                    "class": 0,
                    "name": "person",
                    "confidence": 0.92,
                    "box": { "x1": 100, "y1": 50, "x2": 300, "y2": 400 }
                },
                {
                    "class": 2,
                    "name": "car",
                    "confidence": 0.87,
                    "box": { "x1": 400, "y1": 200, "x2": 600, "y2": 350 }
                }
            ],
            "speed": {
                "preprocess": 1.2,
                "inference": 12.5,
                "postprocess": 2.3
            }
        }
    ],
    "metadata": {
        "imageCount": 1,
        "functionTimeCall": 0.018,
        "task": "detect",
        "version": {
            "ultralytics": "8.x.x",
            "torch": "2.6.0",
            "torchvision": "0.21.0",
            "python": "3.13.0"
        }
    }
}

<!-- screenshot -->

Response Fields

FieldTypeDescription
imagesarrayList of processed images
images[].shapearrayImage dimensions [height, width]
images[].resultsarrayList of detections
images[].results[].classintClass index (integer ID)
images[].results[].namestringClass name
images[].results[].confidencefloatDetection confidence (0-1)
images[].results[].boxobjectBounding box coordinates
images[].speedobjectProcessing times in milliseconds
metadataobjectRequest metadata and version info

Task-Specific Responses

Response format varies by task:

=== "Detection"

```json
{
  "class": 0,
  "name": "person",
  "confidence": 0.92,
  "box": {"x1": 100, "y1": 50, "x2": 300, "y2": 400}
}
```

=== "Segmentation"

```json
{
  "class": 0,
  "name": "person",
  "confidence": 0.92,
  "box": {"x1": 100, "y1": 50, "x2": 300, "y2": 400},
  "segments": {"x": [100, 150, ...], "y": [50, 60, ...]}
}
```

=== "Semantic"

```json
{
  "results": [
    {"class": 0, "name": "road", "pixel_ratio": 0.42},
    {"class": 1, "name": "building", "pixel_ratio": 0.23}
  ]
}
```

Semantic segmentation returns per-class pixel coverage (`pixel_ratio`, the fraction of image pixels assigned to each class) instead of per-object boxes.

=== "Depth"

```json
{
  "results": [],
  "depth": {
    "shape": [480, 640],
    "encoding": "png",
    "data": "<base64 grayscale PNG>",
    "min": 0.31,
    "max": 79.9,
    "bits": 8
  }
}
```

[Depth estimation](../../tasks/depth.md) returns a dense per-pixel map instead of per-object results: a base64-encoded grayscale PNG where `depth = pixel × max / divisor` and a pixel value of `0` means no depth. The optional `bits` request parameter selects the quantization — `8` (default, uint8 PNG, divisor 255), `12`, or `16` (uint16 PNG, divisor 65535). The map is returned at model inference resolution (`imgsz`), so resize it to the image dimensions if you need per-pixel alignment. Decode it with any image library:

```python
import base64
import io

import numpy as np
from PIL import Image

depth = response["images"][0]["depth"]
pixels = np.asarray(Image.open(io.BytesIO(base64.b64decode(depth["data"]))))
meters = pixels * depth["max"] / (255.0 if depth["bits"] == 8 else 65535.0)  # 0 = no depth
```

=== "Pose"

```json
{
  "class": 0,
  "name": "person",
  "confidence": 0.92,
  "box": {"x1": 100, "y1": 50, "x2": 300, "y2": 400},
  "keypoints": {
    "x": [200, ...],
    "y": [75, ...],
    "visible": [0.95, ...]
  }
}
```

The `visible` array contains per-keypoint confidence scores (0-1 floats), not COCO-style 0-2 visibility flags.

=== "Classification"

```json
{
  "results": [
    {"class": 0, "name": "cat", "confidence": 0.95},
    {"class": 1, "name": "dog", "confidence": 0.03}
  ]
}
```

=== "OBB"

```json
{
  "class": 0,
  "name": "ship",
  "confidence": 0.89,
  "box": {
    "x1": 105,
    "y1": 48,
    "x2": 295,
    "y2": 55,
    "x3": 290,
    "y3": 395,
    "x4": 110,
    "y4": 402
  }
}
```

Rate Limits

The shared model API is limited to 20 requests/minute for each API key, signed-in caller, or anonymous IP. When throttled, the API returns 429 with a Retry-After header. See the full rate-limit reference for all endpoint categories.

!!! tip "Need More Throughput?"

Requests sent directly to a [dedicated endpoint](endpoints.md) do not pass through the Platform API rate limiter.
For high-volume local inference, see the [Predict mode guide](../../modes/predict.md).

Error Handling

Common error responses:

CodeMessageSolution
400Invalid imageCheck file format
401UnauthorizedVerify API key
404Model not foundCheck model ID
429Rate limitedWait and retry, or send requests directly to a dedicated endpoint
500Server errorRetry request
503Service unavailablePredict service starting up or unreachable; wait briefly and retry

FAQ

Can I run inference on video?

Both inference methods accept video files:

  • Dedicated endpoints accept video files directly. Supported formats (up to 100 MB): ASF, AVI, GIF, M4V, MKV, MOV, MP4, MPEG, MPG, TS, WEBM, WMV. Each frame is processed individually and results are returned per frame. See dedicated endpoints for details.
  • Shared inference (/api/models/{id}/predict) uses the same predict service and accepts the same video formats. The browser Predict tab only selects images, so use the API or a dedicated endpoint for video.

How do I get the annotated image?

The API returns JSON predictions. To visualize:

  1. Use predictions to draw boxes locally
  2. Use Ultralytics plot() method:
python
from ultralytics import YOLO

model = YOLO("yolo26n.pt")
results = model("image.jpg")
results[0].save("annotated.jpg")

See the Predict mode documentation for the full results API and visualization options.

What's the maximum image size?

  • Predict tab limit: 10 MB
  • Dedicated endpoint API limit: 100 MB
  • Auto-resize in the Predict tab: Images are resized to the selected Image Size before upload

Large images are automatically resized while preserving aspect ratio.

Can I run batch inference?

The current API processes one image per request. For batch:

  1. Send separate requests for each image
  2. Distribute requests across dedicated endpoints when appropriate
  3. Use local inference for large batches

!!! example "Batch Inference with Python"

```python
import concurrent.futures

import requests

url = "https://predict-abc123.run.app/predict"
headers = {"Authorization": "Bearer YOUR_API_KEY"}
images = ["img1.jpg", "img2.jpg", "img3.jpg"]


def predict(image_path):
    with open(image_path, "rb") as f:
        return requests.post(url, headers=headers, files={"file": f}).json()


with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
    results = list(executor.map(predict, images))
```