Voice Activity Detection

+++ disableToc = false title = "Voice Activity Detection (VAD)" weight = 35 url = "/features/voice-activity-detection/" +++

Voice Activity Detection (VAD) identifies segments of speech in audio data. LocalAI provides a /v1/vad endpoint powered by the Silero VAD backend.

The [audio.cpp backend]({{%relref "features/audio-cpp" %}}) also serves this endpoint, and ships the silero_vad and marblenet_vad assets inside its own package, so VAD works there with nothing to download (model: bundled:silero_vad plus the family:silero_vad option).

API

• Method: POST
• Endpoints: /v1/vad, /vad

Request

The request body is JSON with the following fields:

Response

Returns a JSON object with detected speech segments:

Usage

Example request

The /v1/vad endpoint expects the audio field to be an array of raw 16kHz mono PCM samples as float32 values, so the request body is usually built from a real audio file rather than typed by hand.

First convert any audio file to 16kHz mono with ffmpeg:

ffmpeg -i input.mp3 -ar 16000 -ac 1 -f wav speech.wav

Then load the samples and POST them (this snippet needs pip install soundfile numpy requests):

import soundfile as sf
import numpy as np
import requests

audio, sample_rate = sf.read("speech.wav")
if audio.ndim > 1:
    audio = audio.mean(axis=1)  # downmix to mono
samples = audio.astype(np.float32).tolist()

response = requests.post(
    "http://localhost:8080/v1/vad",
    json={"model": "silero-vad", "audio": samples},
)
print(response.json())

Example response

{
  "segments": [
    {
      "start": 0.5,
      "end": 2.3
    },
    {
      "start": 3.1,
      "end": 5.8
    }
  ]
}

Model Configuration

Create a YAML configuration file for the VAD model:

name: silero-vad
backend: silero-vad

Detection Parameters

The Silero VAD backend uses the following internal defaults:

• Sample rate: 16kHz
• Threshold: 0.5
• Min silence duration: 100ms
• Speech pad duration: 30ms

Error Responses