ContextQMD
Back to Open Notebook

oMLX Setup Guide

docs/5-CONFIGURATION/omlx.md

1.14.04.3 KB
Original Source

oMLX Setup Guide

oMLX is a macOS-native inference server for Apple Silicon. It runs MLX models locally and exposes an OpenAI-compatible API at /v1, so Open Notebook can use it for language and embedding models without sending data to the cloud.

Open Notebook treats oMLX as a first-class provider in Settings → API Keys. Identity comes from Esperanto’s built-in omlx OpenAI-compatible profile (AIFactory.create_language("omlx", ...) / create_embedding("omlx", ...)). There is no remapping to the generic openai-compatible provider and no OPENAI_COMPATIBLE_* env mirroring.

Why Choose oMLX?

  • Apple Silicon optimized — MLX inference on M-series Macs
  • OpenAI-compatible — drop-in /v1/chat/completions and /v1/embeddings
  • Private — models and prompts stay on your machine
  • Optional API key — protect the local server with --api-key if needed

Requirements

  • Apple Silicon Mac (M1 or later)
  • macOS 15+ (Sequoia) recommended
  • oMLX installed and running on the host (oMLX cannot run inside Linux containers)

Install oMLX

DMG (recommended): download from oMLX Releases, drag to Applications, and launch.

Homebrew:

bash
brew tap jundot/omlx
brew install omlx
brew services start omlx

CLI:

bash
omlx serve
# or with a custom port (recommended — see Port Conflict below):
OMLX_PORT=11435 omlx serve

Admin UI (when using oMLX’s own default port): http://localhost:8000/admin

Port Conflict with SurrealDB

oMLX and SurrealDB both default to port 8000. Open Notebook’s database uses 8000, so running oMLX on the default port will collide.

Recommended: run oMLX on 11435 (same idea as Ollama’s 11434):

bash
OMLX_PORT=11435 omlx serve

Or set the port in oMLX’s admin settings / ~/.omlx/settings.json.

Then use this base URL in Open Notebook:

text
http://localhost:11435/v1

Always include the /v1 suffix — oMLX’s OpenAI API lives under /v1 (e.g. GET /v1/models).

Configure Open Notebook

  1. Go to Settings → API Keys
  2. Click Add Credential → select oMLX
  3. Set Base URL to http://localhost:11435/v1 (prefilled by default)
  4. Optionally set an API key if you started oMLX with --api-key
  5. Save → Test Connection → Discover Models → register language and embedding models

Legacy env vars (optional fallback)

bash
# Prefer Settings UI; env is for migration / headless setups
export OMLX_API_BASE=http://localhost:11435/v1
# Optional, only if oMLX was started with --api-key
# export OMLX_API_KEY=your-key

These match Esperanto’s profile env names (OMLX_API_BASE, OMLX_API_KEY).

Network Notes (Docker)

oMLX itself must run on the Mac host. If Open Notebook runs in Docker:

SetupBase URL
Both on hosthttp://localhost:11435/v1
Open Notebook in Docker, oMLX on hosthttp://host.docker.internal:11435/v1

Ensure oMLX listens on an interface Docker can reach.

Modalities

ModalitySupported
Language✅
Embedding✅
Speech-to-text❌
Text-to-speech❌

For local STT/TTS, see Local STT and Local TTS (e.g. Speaches via OpenAI-Compatible).

Troubleshooting

SymptomWhat to check
Connection refusedIs oMLX running? Correct port?
Wrong port / SurrealDB errorsDon’t use 8000 for oMLX — use 11435 (or another free port)
404 on /modelsBase URL must end with /v1
401 UnauthorizedAPI key mismatch with oMLX --api-key
No models listedLoad/download models in the oMLX admin UI first
Docker can’t reach hostUse host.docker.internal and confirm oMLX bind address

Quick connectivity check:

bash
curl http://localhost:11435/v1/models