README ONNX - Mnn — ContextQMD

Supported functions

Speech recognition	Speech synthesis
✔️	✔️

Speaker identification	Speaker diarization	Speaker verification
✔️	✔️	✔️

Spoken Language identification	Audio tagging	Voice activity detection
✔️	✔️	✔️

Keyword spotting	Add punctuation	Speech enhancement
✔️	✔️	✔️

Supported platforms

Architecture	Android	iOS	Windows	macOS	linux	HarmonyOS
x64	✔️		✔️	✔️	✔️	✔️
x86	✔️		✔️
arm64	✔️	✔️	✔️	✔️	✔️	✔️
arm32	✔️				✔️	✔️
riscv64					✔️

Supported programming languages

1. C++	2. C	3. Python	4. JavaScript
✔️	✔️	✔️	✔️

5. Java	6. C#	7. Kotlin	8. Swift
✔️	✔️	✔️	✔️

9. Go	10. Dart	11. Rust	12. Pascal
✔️	✔️	✔️	✔️

For Rust support, please see sherpa-rs

It also supports WebAssembly.

Introduction

This repository supports running the following functions locally

Speech-to-text (i.e., ASR); both streaming and non-streaming are supported
Text-to-speech (i.e., TTS)
Speaker diarization
Speaker identification
Speaker verification
Spoken language identification
Audio tagging
VAD (e.g., silero-vad)
Keyword spotting

on the following platforms and operating systems:

x86, x86_64, 32-bit ARM, 64-bit ARM (arm64, aarch64), RISC-V (riscv64)
Linux, macOS, Windows, openKylin
Android, WearOS
iOS
HarmonyOS
NodeJS
WebAssembly
NVIDIA Jetson Orin NX (Support running on both CPU and GPU)
NVIDIA Jetson Nano B01 (Support running on both CPU and GPU)
Raspberry Pi
RV1126
LicheePi4A
VisionFive 2
旭日X3派
爱芯派
etc

with the following APIs

C++, C, Python, Go, C#
Java, Kotlin, JavaScript
Swift, Rust
Dart, Object Pascal

Links for Huggingface Spaces

<details> <summary>You can visit the following Huggingface spaces to try sherpa-onnx without installing anything. All you need is a browser.</summary>

Description	URL
Speaker diarization	Click me
Speech recognition	Click me
Speech recognition with Whisper	Click me
Speech synthesis	Click me
Generate subtitles	Click me
Audio tagging	Click me
Spoken language identification with Whisper	Click me

We also have spaces built using WebAssembly. They are listed below:

Description	Huggingface space	ModelScope space
Voice activity detection with silero-vad	Click me	地址
Real-time speech recognition (Chinese + English) with Zipformer	Click me	地址
Real-time speech recognition (Chinese + English) with Paraformer	Click me	地址
Real-time speech recognition (Chinese + English + Cantonese) with Paraformer-large	Click me	地址
Real-time speech recognition (English)	Click me	地址
VAD + speech recognition (Chinese + English + Korean + Japanese + Cantonese) with SenseVoice	Click me	地址
VAD + speech recognition (English) with Whisper tiny.en	Click me	地址
VAD + speech recognition (English) with Moonshine tiny	Click me	地址
VAD + speech recognition (English) with Zipformer trained with GigaSpeech	Click me	地址
VAD + speech recognition (Chinese) with Zipformer trained with WenetSpeech	Click me	地址
VAD + speech recognition (Japanese) with Zipformer trained with ReazonSpeech	Click me	地址
VAD + speech recognition (Thai) with Zipformer trained with GigaSpeech2	Click me	地址
VAD + speech recognition (Chinese 多种方言) with a TeleSpeech-ASR CTC model	Click me	地址
VAD + speech recognition (English + Chinese, 及多种中文方言) with Paraformer-large	Click me	地址
VAD + speech recognition (English + Chinese, 及多种中文方言) with Paraformer-small	Click me	地址
Speech synthesis (English)	Click me	地址
Speech synthesis (German)	Click me	地址
Speaker diarization	Click me	地址

</details>

Links for pre-built Android APKs

<details> <summary>You can find pre-built Android APKs for this repository in the following table</summary>

Description	URL	中国用户
Speaker diarization	Address	点此
Streaming speech recognition	Address	点此
Text-to-speech	Address	点此
Voice activity detection (VAD)	Address	点此
VAD + non-streaming speech recognition	Address	点此
Two-pass speech recognition	Address	点此
Audio tagging	Address	点此
Audio tagging (WearOS)	Address	点此
Speaker identification	Address	点此
Spoken language identification	Address	点此
Keyword spotting	Address	点此

</details>

Links for pre-built Flutter APPs

Real-time speech recognition

Description	URL	中国用户
Streaming speech recognition	Address	点此

Text-to-speech

Description	URL	中国用户
Android (arm64-v8a, armeabi-v7a, x86_64)	Address	点此
Linux (x64)	Address	点此
macOS (x64)	Address	点此
macOS (arm64)	Address	点此
Windows (x64)	Address	点此

Note: You need to build from source for iOS.

</details>

Links for pre-built Lazarus APPs

Generating subtitles

Description	URL	中国用户
Generate subtitles (生成字幕)	Address	点此

</details>

Links for pre-trained models

Description	URL
Speech recognition (speech to text, ASR)	Address
Text-to-speech (TTS)	Address
VAD	Address
Keyword spotting	Address
Audio tagging	Address
Speaker identification (Speaker ID)	Address
Spoken language identification (Language ID)	See multi-lingual Whisper ASR models from Speech recognition
Punctuation	Address
Speaker segmentation	Address
Speech enhancement	Address

</details>

Some pre-trained ASR models (Streaming)

Please see

for more models. The following table lists only SOME of them.

Name	Supported Languages	Description
sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20	Chinese, English	See also
sherpa-onnx-streaming-zipformer-small-bilingual-zh-en-2023-02-16	Chinese, English	See also
sherpa-onnx-streaming-zipformer-zh-14M-2023-02-23	Chinese	Suitable for Cortex A7 CPU. See also
sherpa-onnx-streaming-zipformer-en-20M-2023-02-17	English	Suitable for Cortex A7 CPU. See also
sherpa-onnx-streaming-zipformer-korean-2024-06-16	Korean	See also
sherpa-onnx-streaming-zipformer-fr-2023-04-14	French	See also

</details>

Some pre-trained ASR models (Non-Streaming)

Please see

for more models. The following table lists only SOME of them.

Name	Supported Languages	Description
Whisper tiny.en	English	See also
Moonshine tiny	English	See also
sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17	Chinese, Cantonese, English, Korean, Japanese	支持多种中文方言. See also
sherpa-onnx-paraformer-zh-2024-03-09	Chinese, English	也支持多种中文方言. See also
sherpa-onnx-zipformer-ja-reazonspeech-2024-08-01	Japanese	See also
sherpa-onnx-nemo-transducer-giga-am-russian-2024-10-24	Russian	See also
sherpa-onnx-nemo-ctc-giga-am-russian-2024-10-24	Russian	See also
sherpa-onnx-zipformer-ru-2024-09-18	Russian	See also
sherpa-onnx-zipformer-korean-2024-06-24	Korean	See also
sherpa-onnx-zipformer-thai-2024-06-20	Thai	See also
sherpa-onnx-telespeech-ctc-int8-zh-2024-06-04	Chinese	支持多种方言. See also