Unsloth Documentation

Unsloth Docs: Train your own model with Unsloth, an open-source framework for LLM fine-tuning and reinforcement learning.
Beginner? Start here!
Unsloth Requirements: Here are Unsloth's requirements including system and GPU VRAM requirements.
FAQ + Is Fine-tuning Right For Me?: If you're stuck on if fine-tuning is right for you, see here! Learn about fine-tuning misconceptions, how it compared to RAG and more:
Unsloth Notebooks: Explore our catalog of Unsloth notebooks:
All Our Models
Install & Update: Learn to install Unsloth locally or online.
Updating: To update or use an old version of Unsloth, follow the steps below:
Pip Install: To install Unsloth locally via Pip, follow the steps below:
Docker: Install Unsloth using our official Docker container
Windows Installation: See how to install Unsloth on Windows with or without WSL.
AMD: Fine-tune with Unsloth on AMD GPUs.
Conda Install: To install Unsloth locally on Conda, follow the steps below:
Google Colab: To install and run Unsloth on Google Colab, follow the steps below:
Fine-tuning LLMs Guide: Learn all the basics and best practices of fine-tuning. Beginner-friendly.
What Model Should I Use?
Datasets Guide: Learn how to create & prepare a dataset for fine-tuning.
LoRA Hyperparameters Guide: Optimal lora rank. alpha, number of epochs, batch size & gradient accumulation, QLoRA vs LoRA, target modules and more!
Tutorial: How to Finetune Llama-3 and Use In Ollama: Beginner's Guide for creating a customized personal assistant (like ChatGPT) to run locally on Ollama
Reinforcement Learning (RL) Guide: Learn all about Reinforcement Learning (RL) and how to train your own DeepSeek-R1 reasoning model with Unsloth using GRPO. A complete guide from beginner to advanced.
Tutorial: Train your own Reasoning model with GRPO: Beginner's Guide to transforming a model like Llama 3.1 (8B) into a reasoning model by using Unsloth and GRPO.
Advanced RL Documentation: Advanced documentation settings when using Unsloth with GRPO.
Memory Efficient RL
RL Reward Hacking: Learn what is Reward Hacking in Reinforcement Learning and how to counter it.
GSPO Reinforcement Learning: Train with GSPO (Group Sequence Policy Optimization) RL in Unsloth.
Reinforcement Learning - DPO, ORPO & KTO: To use the reward modelling functions for DPO, GRPO, ORPO or KTO with Unsloth, follow the steps below:
DeepSeek-OCR: How to Run & Fine-tune: Guide on how to run and fine-tune DeepSeek-OCR locally.
How to Fine-tune LLMs with Unsloth & Docker: Learn how to fine-tune LLMs or do Reinforcement Learning (RL) with Unsloth's Docker image.
Vision Reinforcement Learning (VLM RL): Train Vision/multimodal models via GRPO and RL with Unsloth!
gpt-oss Reinforcement Learning
Tutorial: How to Train gpt-oss with RL: Learn to train OpenAI gpt-oss with GRPO to autonomously beat 2048 locally or on Colab.
Unsloth Dynamic GGUFs on Aider Polyglot: Performance of Unsloth Dynamic GGUFs on Aider Polyglot Benchmarks
Qwen3-VL: How to Run & Fine-tune: Learn to fine-tune and run Qwen3-VL locally with Unsloth.
gpt-oss: How to Run & Fine-tune: Run & fine-tune OpenAI's new open-source models!
Tutorial: How to Fine-tune gpt-oss: Learn step-by-step how to train OpenAI gpt-oss locally with Unsloth.
Long Context gpt-oss Training
GLM-4.6: How to Run Locally: A guide on how to run Z.ai's new GLM-4.6 model on your own local device!
IBM Granite 4.0: How to run IBM Granite-4.0 with Unsloth GGUFs on llama.cpp, Ollama and how to fine-tune!
DeepSeek-V3.1: How to Run Locally: A guide on how to run DeepSeek-V3.1 and Terminus on your own local device!
Qwen3-Coder: How to Run Locally: Run Qwen3-Coder-30B-A3B-Instruct and 480B-A35B locally with Unsloth Dynamic quants.
Gemma 3: How to Run & Fine-tune: How to run Gemma 3 effectively with our GGUFs on llama.cpp, Ollama, Open WebUI and how to fine-tune with Unsloth!
Gemma 3n: How to Run & Fine-tune: Run Google's new Gemma 3n locally with Dynamic GGUFs on llama.cpp, Ollama, Open WebUI and fine-tune with Unsloth!
Qwen3: How to Run & Fine-tune: Learn to run & fine-tune Qwen3 locally with Unsloth + our Dynamic 2.0 quants
Qwen3-2507: Run Qwen3-30B-A3B-2507 and 235B-A22B Thinking and Instruct versions locally on your device!
Tutorials: How To Fine-tune & Run LLMs: Learn how to run and fine-tune models for optimal performance 100% locally with Unsloth.
DeepSeek-R1-0528: How to Run Locally: A guide on how to run DeepSeek-R1-0528 including Qwen3 on your own local device!
Magistral: How to Run & Fine-tune: Meet Magistral - Mistral's new reasoning models.
Llama 4: How to Run & Fine-tune: How to run Llama 4 locally using our dynamic GGUFs which recovers accuracy compared to standard quantization.
Kimi K2: How to Run Locally: Guide on running Kimi K2 and Kimi-K2-Instruct-0905 on your own local device!
Grok 2: Run xAI's Grok 2 model locally!
Devstral: How to Run & Fine-tune: Run and fine-tune Mistral Devstral 1.1, including Small-2507 and 2505.
DeepSeek-V3-0324: How to Run Locally: How to run DeepSeek-V3-0324 locally using our dynamic quants which recovers accuracy
DeepSeek-R1: How to Run Locally: A guide on how you can run our 1.58-bit Dynamic Quants for DeepSeek-R1 using llama.cpp.
DeepSeek-R1 Dynamic 1.58-bit: See performance comparison tables for Unsloth's Dynamic GGUF Quants vs Standard IMatrix Quants.
QwQ-32B: How to Run effectively: How to run QwQ-32B effectively with our bug fixes and without endless generations + GGUFs.
Phi-4 Reasoning: How to Run & Fine-tune: Learn to run & fine-tune Phi-4 reasoning models locally with Unsloth + our Dynamic 2.0 quants
Running & Saving Models: Learn how to save your finetuned model so you can run it in your favorite inference engine.
Saving to GGUF: Saving models to 16bit for GGUF so you can use it for Ollama, Jan AI, Open WebUI and more!
Saving to Ollama
Saving to vLLM for deployment: Saving models to 16bit for vLLM deployment and serving
Saving to SGLang for deployment: Saving models to 16bit for SGLang for deployment and serving
Unsloth Inference: Learn how to run your finetuned model with Unsloth's faster inference.
Troubleshooting Inference: If you're experiencing issues when running or saving your model.
vLLM Engine Arguments
LoRA Hot Swapping Guide
Text-to-Speech (TTS) Fine-tuning: Learn how to fine-tune TTS & STT voice models with Unsloth.
Unsloth Dynamic 2.0 GGUFs: A big new upgrade to our Dynamic Quants!
Vision Fine-tuning: Learn how to fine-tune vision/multimodal LLMs with Unsloth
Fine-tuning LLMs with NVIDIA DGX Spark and Unsloth: Tutorial on how to fine-tune and do reinforcement learning (RL) with OpenAI gpt-oss on NVIDIA DGX Spark.
Fine-tuning LLMs with Blackwell, RTX 50 series & Unsloth: Learn how to fine-tune LLMs on NVIDIA's Blackwell RTX 50 series and B200 GPUs with our step-by-step guide.
Multi-GPU Training with Unsloth: Learn how to fine-tune LLMs on multiple GPUs and parallelism with Unsloth.
Finetuning from Last Checkpoint: Checkpointing allows you to save your finetuning progress so you can pause it and then continue.
Troubleshooting & FAQs: Tips to solve issues, and frequently asked questions.
Chat Templates: Learn the fundamentals and customization options of chat templates, including Conversational, ChatML, ShareGPT, Alpaca formats, and more!
Quantization-Aware Training (QAT): Quantize models to 4-bit with Unsloth and PyTorch to recover accuracy.
Unsloth Environment Flags: Advanced flags which might be useful if you see breaking finetunes, or you want to turn stuff off.
Continued Pretraining: AKA as Continued Finetuning. Unsloth allows you to continually pretrain so a model can learn a new language.
Unsloth Benchmarks: Unsloth recorded benchmarks on NVIDIA GPUs.