tauri-examples/non-streaming-speech-recognition-from-file/README.md
A Tauri v2 desktop app that transcribes audio and video files using offline ASR with Silero VAD.
Pre-built apps (macOS, Linux, Windows) are available at:
| 1 | 2 | 3 |
|---|---|---|
https://www.bilibili.com/video/BV1cXoKBhEdz
.srt filesAny format supported by symphonia:
| Type | Formats |
|---|---|
| Audio | MP3, FLAC, AAC, OGG/Vorbis, WAV, AIFF, CAF, ADPCM |
| Video | MP4/M4A, MKV, WebM (audio track extracted) |
Install npm dependencies:
npm install
This example bundles the SenseVoice int8 model (model type 15), which supports Chinese, English, Japanese, Korean, and Cantonese.
cd tauri-examples/non-streaming-speech-recognition-from-file
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17.tar.bz2
tar xvf sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17.tar.bz2
rm sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17.tar.bz2
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx
Tauri bundles files from src-tauri/assets/. Place the model directory
(keeping its original name) and silero_vad.onnx inside it:
mkdir -p src-tauri/assets
cp -a sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17 src-tauri/assets/
cp silero_vad.onnx src-tauri/assets/
Expected layout:
src-tauri/assets/
├── silero_vad.onnx
└── sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17/
├── model.int8.onnx
└── tokens.txt
Some models (e.g. SenseVoice) support a homophone replacer for correcting
commonly confused words. To enable it, place lexicon.txt and replace.fst
directly inside assets/:
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/hr-files/lexicon.txt
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/hr-files/replace.fst
cp lexicon.txt replace.fst src-tauri/assets/
The app works fine without these files.
npm run dev
This opens the app window. Click Select Audio/Video File, pick a file, and recognition starts automatically.
npm run build
The output is in src-tauri/target/release/bundle/.
The app uses two constants in src-tauri/src/lib.rs to select which model
to bundle:
const MODEL_TYPE: u32 = 15;
const MODEL_NAME: &str = "sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17";
To switch models:
src-tauri/src/model_registry.rs (types 0--61).MODEL_TYPE and MODEL_NAME in src-tauri/src/lib.rs.src-tauri/assets/, keeping the original directory name.npm run dev to test.You can also use the build script generator to automate this:
python scripts/tauri/generate-vad-asr.py --gen-registry
python scripts/tauri/generate-vad-asr.py --total 1 --index 0
┌──────────────────────────────────────────────────┐
│ Frontend (HTML + JS) │
│ index.html / main.js / styles.css │
│ │
│ invoke("recognize_file", {path}) │
│ setInterval → invoke("get_recognition_progress")│
└──────────────┬───────────────────────────────────┘
│ Tauri IPC (invoke)
┌──────────────▼───────────────────────────────────┐
│ Backend (Rust) │
│ │
│ lib.rs │
│ ├── recognize_file() → spawns worker thread │
│ ├── run_recognition() → decode → VAD → ASR │
│ ├── get_recognition_progress() → poll state │
│ ├── cancel_recognition() │
│ ├── export_srt() │
│ └── save_segment_as_wav() │
│ │
│ model_registry.rs (auto-generated, 62 models) │
└──────────────────────────────────────────────────┘
Processing pipeline:
| Problem | Solution |
|---|---|
| "Models are still loading" on startup | Models load in a background thread. Wait a few seconds. Large models take longer. |
| "Failed to create recognizer" | Check that src-tauri/assets/<MODEL_NAME>/ exists and contains the expected .onnx and tokens.txt files. |
| "Unsupported format" | The audio format may not be enabled in Cargo.toml's symphonia features. Check the [dependencies.symphonia] section. |
| Black rectangle in player | Normal for audio-only files — the <video> element shows controls but no picture. |
| App crashes on startup | Run npm run dev and check stderr for [init] or [build_models] log lines. |