Back to Mistral Rs

mistralrs Python SDK

mistralrs-pyo3/README.md

0.8.0538 B
Original Source

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

bash
pip install mistralrs-cuda   # NVIDIA GPUs
pip install mistralrs-metal  # Apple Silicon
pip install mistralrs        # CPU only