mistralrs Python SDK

mistralrs-pyo3/README.md

0.8.0538 B

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

bash

pip install mistralrs-cuda   # NVIDIA GPUs
pip install mistralrs-metal  # Apple Silicon
pip install mistralrs        # CPU only