GenieX Python Library Inference Examples

Run LLM and VLM inference on Windows on Snapdragon from Python.

Prerequisites

ARM64 Python 3.x. Do not use conda or x86 Python (run conda deactivate if conda is active). Step 1 below checks your current install; if it fails, Step 1a can download and launch the official installer for you.

Already have ARM64 Python 3.x? Skip Step 1 / 1a / 1b and jump straight to Step 2.

Step 1: Check Python on PATH

Verify that the python on PATH is ARM64.

python

import subprocess

probe = "import sys, platform; print(sys.version_info.major, sys.version_info.minor, sys.version_info.micro, platform.machine())"
r = subprocess.run(["python", "-c", probe], capture_output=True, text=True)

if r.returncode != 0:
    print("✗ No 'python' found on PATH. Run Step 1a to install.")
else:
    major, minor, micro, arch = r.stdout.split()
    ok = arch.lower() in ("arm64", "aarch64")
    print(f"{'✓' if ok else '✗'} Python {major}.{minor}.{micro} {arch}")
    if not ok:
        print("  Need ARM64 Python. Run Step 1a to install it, or Step 1b if ARM64 is already installed but shadowed by conda / x86 Python.")

Step 1a: Install Python ARM64 (only if Step 1 failed)

Downloads the official Python 3.13 ARM64 installer and launches it. In the installer window, check "Add python.exe to PATH" and click Install Now. Wait for the installer window to close, then restart VS Code / Jupyter and re-run Step 1.

python

!powershell -NoProfile -Command "$p = \"$env:TEMP\python-arm64-installer.exe\"; Invoke-WebRequest https://www.python.org/ftp/python/3.13.3/python-3.13.3-arm64.exe -OutFile $p; Start-Process $p"

Step 1b: Fix PATH (only if Step 1 failed)

Restore the system PATH for the current kernel session. If the check above still fails, open System Environment Variables → edit Path → move ARM64 Python to the top. (This cell only affects the current session; restart the kernel → re-run it.)

python

!powershell -NoProfile -Command "$systemPath = [Environment]::GetEnvironmentVariable('Path', 'Machine'); $userPath = [Environment]::GetEnvironmentVariable('Path', 'User'); [Environment]::SetEnvironmentVariable('Path', \"$userPath;$systemPath\", 'Process')"

Step 2: Create a virtual environment

python

!python -m venv --clear geniex-env

python

import subprocess

probe = "import sys, platform; print(f'{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro} {platform.machine()}')"
r = subprocess.run([r"geniex-env\Scripts\python", "-c", probe], capture_output=True, text=True)

if r.returncode != 0:
    print("✗ Venv not found. Run the cell above first.")
else:
    info = r.stdout.strip()
    ok = "ARM64" in info.upper()
    print(f"{'✓' if ok else '✗'} Venv Python: {info}")
    if not ok:
        print("  Wrong arch. Fix PATH in Step 1b, then re-run Step 2.")

Step 3: Select the venv as the Jupyter kernel

Click the kernel picker (top-right) → Python Environments → pick geniex-env (or browse to geniex-env\Scripts\python.exe). Then run the cell below to confirm the active kernel.

python

import sys, platform

ver = sys.version_info
arch = platform.machine()
ok = arch.lower() == "arm64"
print(f"{'✓' if ok else '✗'} Kernel: Python {ver.major}.{ver.minor}.{ver.micro} {arch}")
if not ok:
    print("  Kernel is not ARM64. Re-select the geniex-env kernel.")

Step 4: Install GenieX into the venv

Run this only after Step 3 verification passed, so %pip installs into geniex-env and not the original kernel.

python

%pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple "geniex[notebook]"

Example 1: LLM text generation

Model weights are downloaded on first use.

python

from geniex import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "bartowski/Qwen_Qwen3.5-0.8B-GGUF",
    precision="Q4_0",
    device_map="llama_cpp"
)

messages = [{"role": "user", "content": "What is 2+2?"}]
prompt = model.tokenizer.apply_chat_template(
    messages, add_generation_prompt=True,
)

streamer = model.generate(prompt, max_new_tokens=256, stream=True)
for chunk in streamer:
    print(chunk, end="", flush=True)
print("\n\n", streamer.output.profile)

model.close()

Example 2: VLM image description

Downloads demo.jpg next to this notebook, then runs SmolVLM on it.

python

!curl --ssl-no-revoke -Lo demo.jpg https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-geniex/demo.jpg

python

import os
from geniex import AutoModelForCausalLM

image_path = os.path.abspath("demo.jpg")

model = AutoModelForCausalLM.from_pretrained(
    "ggml-org/SmolVLM-500M-Instruct-GGUF",
    precision="Q8_0",
    device_map="llama_cpp",
)

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image_path},
        {"type": "text", "text": "Describe the image."},
    ],
}]
prompt = model.tokenizer.apply_chat_template(
    messages, add_generation_prompt=True,
)

streamer = model.generate(prompt, images=[image_path], max_new_tokens=256, stream=True)
for chunk in streamer:
    print(chunk, end="", flush=True)
print("\n\n", streamer.output.profile)

model.close()