Back to Moonshine

Moonshine Micro — Voice Interfaces for Microcontrollers

micro/README.md

0.0.693.2 KB
Original Source

Moonshine Micro — Voice Interfaces for Microcontrollers

Moonshine Voice is an open source AI toolkit for developers building real-time voice agents and applications. Moonshine Micro is a version designed specifically for embedded system processors like microcontrollers and DSPs, and uses the Raspberry Pi RP2350, which retails for just 80 cents, as its reference platform. It includes voice-activity detection, command recognition, and neural speech synthesis and can run in as little as 470 KB of RAM.

You can see a full walkthrough in the video below:

<p align="center"> <a href="https://www.youtube.com/watch?v=kMliOFYBiz4"> </a> </p>

The memory and compute requirements are designed to fit resource-constrained systems. Figures below are for the RP2350 demo; the detailed memory budget breaks each one down:

ComponentFlashSRAM (arena peak)Compute
VAD (Voice Activity Detection)~89 KiB~36 KiB~0.8 MMAC/frame (~25 MMAC/s)
STT (SpellingCNN Speech-to-Text)~1.3 MiB~346 KiB~36 MMAC/s
TTS (neural diphone synth @ 16 kHz)~1.8 MiB voice pack~340 KiB~37 MMAC typical reply (~65 MMAC/s out)
TOTAL (Demo pipeline)~3.6 MiB~468 KiB provisioned*classify + speak ~0.7–1.0 s

Notes:

  • Flash is .text + .rodata measured with arm-none-eabi-size on the default moonshine_micro_echo firmware (includes the embedded neural voice pack); SRAM is .bss + heap + stacks.
  • *VAD, STT, and neural TTS run sequentially and time-share one ~384 KiB TFLM arena, so SRAM is not additive — ~468 KiB is the total RAM provisioned on the 520 KiB RP2350 (wifi_hardware ~491 KiB).
  • A MAC is one multiply-accumulate; MMAC/s = millions per second during the active (non-idle) stage.

The code is released under the permissive MIT License, suitable for commercial applications.

There's a complete end-to-end example showing how to set up a wifi connection on a microcontroller using voice on an RP2350 MCU.

The VAD, STT, and TTS libraries can be used independently of each other, relying on the included TensorFlow Lite Micro library for the neural computations.

Documentation

License

This code, apart from the source in third-party/, is licensed under the MIT License — see LICENSE in this directory (also at the repository root).

The SpellingCNN and TinyVadCNN models in models/ are released under the MIT License.

The code in third-party/ is licensed according to the terms of the open source projects it originates from, with details in a LICENSE file in each subfolder.