CLI reference

import Feedback from "/snippets/page-feedback.mdx";

Model inference

geniex pull

Download a model and store it locally.

geniex pull <model-name>[:<precision>]

Pulling from a local path:

geniex pull local/my-model --local-path /path/to/model-dir

<Note>pull copies files into the GenieX cache. After a successful pull you can safely delete the source to avoid keeping two copies.</Note>

Precision (Quantization) (llama.cpp only)

For GGUF models the CLI prompts you to pick a precision:

Choose a precision version to download
> Q4_0       [1.2 GiB] (default)
  Q8_0       [2.0 GiB]
  F16        [3.8 GiB]

<Tip>Q4_0 has the best Hexagon NPU support. See Precisions (Quantizations) Supported.</Tip>

Qualcomm AI Hub Models are pre-quantized — no choice needed.

geniex infer — LLM

Launch an interactive chat session with a language model.

geniex infer ai-hub-models/Qwen3-4B

Thinking mode — control whether the model shows reasoning before responding:

geniex infer ai-hub-models/Qwen3-4B --think         # show reasoning steps
geniex infer ai-hub-models/Qwen3-4B --think=false   # respond directly

Compute unit selection (via --compute) — pick which compute unit runs the model (default: npu):

# llama.cpp models support all compute units
geniex infer unsloth/Qwen3.5-0.8B-GGUF --compute npu
geniex infer unsloth/Qwen3.5-0.8B-GGUF --compute gpu
geniex infer unsloth/Qwen3.5-0.8B-GGUF --compute cpu

# Qualcomm AI Hub Models only support NPU
geniex infer ai-hub-models/Qwen3-4B --compute npu

<Warning>Qualcomm AI Hub Models run on NPU only. Using --compute cpu or --compute gpu returns an error.</Warning>

geniex infer — VLM

Run vision-language inference with text-only or image input:

geniex infer ai-hub-models/Qwen2.5-VL-7B-Instruct-GGUF

For text-only, just launch and chat. For image input, provide the absolute path or drag the file into your terminal:

Describe this picture </full/path/to/image.png>

geniex serve

Start the OpenAI-compatible local server. See Local server for the API.

geniex serve

Logging

--log is a global flag controlling the CLI's log output. It is equivalent to the GENIEX_LOG environment variable and takes precedence over it when both are set.

geniex --log debug list

Configuration flags

These flags can be passed to geniex infer to control model loading and generation.

Sampler flags

Control how the model selects tokens during generation.

Model flags

Control model loading, context, and generation limits.

Increasing the context length

The context window (--nctx) is how much the model can hold at once. When a conversation grows past it you get a context length exceeded error. How to raise it depends on the runtime:

• llama.cpp (GGUF): --nctx <N> raises the window at runtime, up to the model's trained maximum. A larger window uses more KV-cache memory.

  bashCopy

  geniex infer unsloth/Qwen3-8B-GGUF --nctx 8192
  

• Qualcomm AI Engine Direct (NPU): the context length is baked into the compiled bundle and cannot be raised at runtime — --nctx has no effect. Instead:

  • Add --sliding-window to keep chatting past the limit: the oldest context above a small anchored prefix is evicted instead of erroring.
  • For a genuinely larger window, pull a bundle compiled for a longer context from Qualcomm AI Hub. See Qualcomm AI Engine Direct runtime constraints.

Utility commands