CLI & Automation

Classic table output, the REST API for cluster scheduling, hardware overrides, context caps, and JSON output for scripts and agents.

← Back to README

CLI mode

Use --cli or any subcommand to get classic table output:

# Table of all models ranked by fit
llmfit --cli

# Only perfectly fitting models, top 5
llmfit fit --perfect -n 5

# Show detected system specs
llmfit system

# Hardware diagnostic report for bug reports (raw nvidia-smi/rocm-smi/sysfs
# output + what llmfit detected) — paste into a GitHub issue
llmfit doctor

# List all models in the database
llmfit list

# Search by name, provider, or size
llmfit search "llama 8b"

# Detailed view of a single model
llmfit info "Mistral-7B"

# Top 5 recommendations (JSON, for agent/script consumption)
llmfit recommend --json --limit 5

# Recommendations filtered by use case
llmfit recommend --json --use-case coding --limit 3

# Force a specific runtime (bypass automatic MLX selection on Apple Silicon)
llmfit recommend --force-runtime llamacpp
llmfit recommend --force-runtime llamacpp --use-case coding --limit 3

# Plan required hardware for a specific model configuration
llmfit plan "Qwen/Qwen3-4B-MLX-4bit" --context 8192
llmfit plan "Qwen/Qwen3-4B-MLX-4bit" --context 8192 --quant mlx-4bit
llmfit plan "Qwen/Qwen3-4B-MLX-4bit" --context 8192 --target-tps 25 --json

# Run as a node-level REST API (for cluster schedulers / aggregators)
llmfit serve --host 0.0.0.0 --port 8787

REST API (llmfit serve)

llmfit serve starts an HTTP API that exposes the same fit/scoring data used by TUI/CLI, including filtering and top-model selection for a node.

# Liveness
curl http://localhost:8787/health

# Node hardware info
curl http://localhost:8787/api/v1/system

# Full fit list with filters
curl "http://localhost:8787/api/v1/models?min_fit=marginal&runtime=llamacpp&sort=score&limit=20"

# Key scheduling endpoint: top runnable models for this node
curl "http://localhost:8787/api/v1/models/top?limit=5&min_fit=good&use_case=coding"

# Search by model name/provider text
curl "http://localhost:8787/api/v1/models/Mistral?runtime=any"

Supported query params for models/models/top:

• limit (or n): max number of rows returned
• perfect: true|false (forces perfect-only when true)
• min_fit: perfect|good|marginal|too_tight
• runtime: any|mlx|llamacpp
• use_case: general|coding|reasoning|chat|multimodal|embedding
• provider: provider text filter (substring)
• search: free-text filter across name/provider/size/use-case
• sort: score|tps|params|mem|ctx|date|use_case
• include_too_tight: include non-runnable rows (default false on /top, true on /models)
• max_context: per-request context cap for memory estimation
• force_runtime: mlx|llamacpp|vllm — override automatic runtime selection during analysis

Validate API behavior locally:

# spawn server automatically and run endpoint/schema/filter assertions
python3 scripts/test_api.py --spawn

# or test an already-running server
python3 scripts/test_api.py --base-url http://127.0.0.1:8787

Contributing benchmarks (bench --share)

llmfit bench measures inference performance against a running provider (Ollama, vLLM, MLX, or llama-server). llama-server is auto-detected on port 8080 via its /props endpoint (override with LLAMA_SERVER_HOST for a full URL, or LLAMA_SERVER_PORT), or select it explicitly with --provider llamacpp. Add --share to contribute your results back to the project as a pull request — no gh CLI and no account on a third-party service required:

# Benchmark every discovered model and open a PR with the results
llmfit bench --all --share

# Preview the exact JSON payloads without contacting GitHub
llmfit bench --all --share --dry-run

# Skip the confirmation prompt (e.g. for automation)
llmfit bench --all --share --yes

# Upload previously stored local benchmarks without benchmarking again
llmfit bench --share

Every successful bench run is also saved locally (under ~/.local/share/llmfit/benchmarks/pending/ on Linux; override the location with LLMFIT_BENCH_STORE), so skipping --share never discards data. These local results appear at the top of the TUI leaderboard as “you (local)”, and they feed back into the fit table: a model you benched shows your measured tok/s instead of the estimate, and runs on trustworthy models (≥ 1B params, dense) calibrate the formula estimates for every other model on the same hardware (shown as “Calibrated ×N from your own llmfit bench run(s)” in the estimate basis). Runs recorded on a different CPU/GPU configuration are ignored. Sharing later — llmfit bench --share on its own, or the share toggle in the TUI — offers to contribute all stored benchmarks in a single PR; uploaded files move to .../benchmarks/shared/ so they are kept as history but never submitted twice.

Merged submissions ship in the next release. Community files are embedded into the binary at build time, so anyone on identical hardware (same CPU + GPU) sees them on the benchmark page as llmfit community rows, gets measured ✓ tok/s for those models, and gets calibrated estimates everywhere else — a fresh install benefits before its user ever runs a benchmark. Trust order everywhere: your own runs > llmfit community on identical hardware > localmaxxing medians on matching presets > formula estimate.

Authentication uses the GitHub device flow (the same mechanism gh auth login uses): llmfit prints a short code and a URL, you approve it in your browser once, and the token is cached under ~/.config/llmfit/ for next time. If a GITHUB_TOKEN or GH_TOKEN environment variable is set (or you use CI), that token is used automatically and no browser step is needed. With --share, credentials are resolved and verified before any benchmark starts, so a missing or expired token fails fast instead of after minutes of benching.

--share then forks the repo, commits one result file per stored submission under llmfit-core/data/community/<hardware>/, and opens a pull request — or, if you already have an open benchmark PR, appends the new results to it instead of opening another. Submissions are idempotent: file names mirror your local store, so retrying after a partial failure skips anything that already landed. Nothing is submitted until you confirm, and --dry-run never touches the network.

Interactive login ships enabled — the public OAuth App client id is baked into the binary (the device flow needs no client secret, so this is safe by design). LLMFIT_GH_CLIENT_ID overrides it (e.g. when running a fork against your own OAuth App); set it to an empty string to disable interactive login entirely and rely on GITHUB_TOKEN / GH_TOKEN.

Hardware overrides

Hardware autodetection can fail on some systems (e.g. broken nvidia-smi, VMs, passthrough setups), or you may want to evaluate model fit against different target hardware. Use --memory, --ram, and --cpu-cores to override detected values:

# Override GPU VRAM
llmfit --memory=32G

# Override system RAM
llmfit --ram=128G

# Override CPU core count
llmfit --cpu-cores=16

# Combine overrides to simulate target hardware
llmfit --memory=24G --ram=64G --cpu-cores=8 fit
llmfit --memory=24G --ram=64G system --json

# Works with all modes: TUI, CLI, and subcommands
llmfit --memory=24G --cli
llmfit --memory=24G fit --perfect -n 5
llmfit --ram=64G recommend --json

Accepted suffixes for --memory and --ram: G/GB/GiB (gigabytes), M/MB/MiB (megabytes), T/TB/TiB (terabytes). Case-insensitive. If no GPU was detected, --memory creates a synthetic GPU entry so models are scored for GPU inference. On unified-memory systems (Apple Silicon), --ram also updates VRAM; use --memory to override VRAM independently.

Context-length cap for estimation

Use --max-context to cap context length used for memory estimation (without changing each model's advertised maximum context):

# Estimate memory fit at 4K context
llmfit --max-context 4096 --cli

# Works with subcommands
llmfit --max-context 8192 fit --perfect -n 5
llmfit --max-context 16384 recommend --json --limit 5

If --max-context is not set, llmfit will use OLLAMA_CONTEXT_LENGTH when available.

JSON output

Add --json to any subcommand for machine-readable output:

llmfit --json system     # Hardware specs as JSON
llmfit --json fit -n 10  # Top 10 fits as JSON
llmfit recommend --json  # Top 5 recommendations (JSON is default for recommend)
llmfit plan "Qwen/Qwen2.5-Coder-0.5B-Instruct" --context 8192 --json

plan JSON includes stable fields for:

• request (context, quantization, target_tps)
• estimated minimum/recommended hardware
• per-path feasibility (gpu, cpu_offload, cpu_only)
• upgrade deltas