ContextQMD
Back to Char

Local Models

apps/web/content/docs/developers/16.local-models.mdx

latest3.1 KB
Original Source

Language models (LLM)

For step-by-step setup instructions for LM Studio and Ollama, see the Local LLM Setup guide.

Speech-to-text (STT)

  • All STT models are stored in ~/Library/Application Support/hyprnote/models/stt/
  • The built-in whisper-cpp backend is now an optional compile-time feature. Production builds on macOS Apple Silicon use the Argmax sidecar by default. Whisper-cpp quantized models have been removed from the settings UI.

Cactus engine (beta)

Cactus is a local STT backend that runs optimized Whisper models. Cactus models are listed in Settings > Transcription under the Hyprnote provider. Models with an "Apple NPU" suffix use the Apple Neural Engine for hardware-accelerated inference on Apple Silicon Macs; non-Apple variants run on any platform.

ModelQuantizationPlatform
Whisper Small (INT4)INT4General
Whisper Small (INT8)INT8General
Whisper Small (INT8, Apple NPU)INT8Apple Silicon
Whisper Medium (INT4)INT4General
Whisper Medium (INT4, Apple NPU)INT4Apple Silicon
Whisper Medium (INT8)INT8General
Whisper Medium (INT8, Apple NPU)INT8Apple Silicon

Cactus models are downloaded and managed from the Settings UI. They are stored in a separate cactus/ directory within the models folder.

Timestamps: Cactus v1.11+ produces word-level timestamps in transcription output, enabling precise alignment of transcript words with audio playback.

Custom vocabulary: Cactus supports custom vocabulary to improve recognition accuracy for domain-specific terms, names, and acronyms. Custom vocabulary words are defined in Settings > Intelligence > Memory and are passed to the Cactus engine during both streaming and batch transcription.

Argmax models

Manual download:

URLFolder
https://huggingface.co/argmaxinc/whisperkit-pro/tree/main/openai_whisper-large-v3-v20240930_626MBopenai_whisper-large-v3-v20240930_626MB
https://huggingface.co/argmaxinc/parakeetkit-pro/tree/main/nvidia_parakeet-v2_476MBnvidia_parakeet-v2_476MB
https://huggingface.co/argmaxinc/parakeetkit-pro/tree/main/nvidia_parakeet-v3_494MBnvidia_parakeet-v3_494MB
https://huggingface.co/argmaxinc/ctckit-pro/tree/main/parakeet-tdt_ctc-110mparakeet-tdt_ctc-110m

For details on how your data is handled when using local vs cloud models, see AI Models & Data Privacy.

If you have Huggingface CLI installed:

bash
hf download argmaxinc/whisperkit-pro \
  --include "openai_whisper-large-v3-v20240930_626MB/*" \
  --local-dir "$HOME/Library/Application Support/hyprnote/models/stt"