Unsloth

Unsloth Skill

Comprehensive assistance with unsloth development, generated from official documentation.

When to Use This Skill

This skill should be triggered when:

• Working with unsloth
• Asking about unsloth features or APIs
• Implementing unsloth solutions
• Debugging unsloth code
• Learning unsloth best practices

Quick Reference

Common Patterns

Quick reference patterns will be added as you use the skill.

Reference Files

This skill includes comprehensive documentation in references/:

• llms-txt.md - Llms-Txt documentation

Use view to read specific reference files when detailed information is needed.

Working with This Skill

For Beginners

Start with the getting_started or tutorials reference files for foundational concepts.

For Specific Features

Use the appropriate category reference file (api, guides, etc.) for detailed information.

For Code Examples

The quick reference section above contains common patterns extracted from the official docs.

Resources

references/

Organized documentation extracted from official sources. These files contain:

• Detailed explanations
• Code examples with language annotations
• Links to original documentation
• Table of contents for quick navigation

scripts/

Add helper scripts here for common automation tasks.

assets/

Add templates, boilerplate, or example projects here.

Notes

• This skill was automatically generated from official documentation
• Reference files preserve the structure and examples from source docs
• Code examples include language detection for better syntax highlighting
• Quick reference patterns are extracted from common usage examples in the docs

Updating

To refresh this skill with updated documentation: 1. Re-run the scraper with the same configuration 2. The skill will be rebuilt with the latest information

<!-- Trigger re-upload 1763621536 -->

Unsloth Documentation Index

Categories

Llms-Txt

File: llms-txt.md Pages: 136

Unsloth Documentation

Unsloth Documentation

• Unsloth Docs: Train your own model with Unsloth, an open-source framework for LLM fine-tuning and reinforcement learning.
• Beginner? Start here!
• Unsloth Requirements: Here are Unsloth's requirements including system and GPU VRAM requirements.
• FAQ + Is Fine-tuning Right For Me?: If you're stuck on if fine-tuning is right for you, see here! Learn about fine-tuning misconceptions, how it compared to RAG and more:
• Unsloth Notebooks: Explore our catalog of Unsloth notebooks:
• All Our Models
• Install & Update: Learn to install Unsloth locally or online.
• Updating: To update or use an old version of Unsloth, follow the steps below:
• Pip Install: To install Unsloth locally via Pip, follow the steps below:
• Docker: Install Unsloth using our official Docker container
• Windows Installation: See how to install Unsloth on Windows with or without WSL.
• AMD: Fine-tune with Unsloth on AMD GPUs.
• Conda Install: To install Unsloth locally on Conda, follow the steps below:
• Google Colab: To install and run Unsloth on Google Colab, follow the steps below:
• Fine-tuning LLMs Guide: Learn all the basics and best practices of fine-tuning. Beginner-friendly.
• What Model Should I Use?
• Datasets Guide: Learn how to create & prepare a dataset for fine-tuning.
• LoRA Hyperparameters Guide: Optimal lora rank. alpha, number of epochs, batch size & gradient accumulation, QLoRA vs LoRA, target modules and more!
• Tutorial: How to Finetune Llama-3 and Use In Ollama: Beginner's Guide for creating a customized personal assistant (like ChatGPT) to run locally on Ollama
• Reinforcement Learning (RL) Guide: Learn all about Reinforcement Learning (RL) and how to train your own DeepSeek-R1 reasoning model with Unsloth using GRPO. A complete guide from beginner to advanced.
• Tutorial: Train your own Reasoning model with GRPO: Beginner's Guide to transforming a model like Llama 3.1 (8B) into a reasoning model by using Unsloth and GRPO.
• Advanced RL Documentation: Advanced documentation settings when using Unsloth with GRPO.
• Memory Efficient RL
• RL Reward Hacking: Learn what is Reward Hacking in Reinforcement Learning and how to counter it.
• GSPO Reinforcement Learning: Train with GSPO (Group Sequence Policy Optimization) RL in Unsloth.
• Reinforcement Learning - DPO, ORPO & KTO: To use the reward modelling functions for DPO, GRPO, ORPO or KTO with Unsloth, follow the steps below:
• DeepSeek-OCR: How to Run & Fine-tune: Guide on how to run and fine-tune DeepSeek-OCR locally.
• How to Fine-tune LLMs with Unsloth & Docker: Learn how to fine-tune LLMs or do Reinforcement Learning (RL) with Unsloth's Docker image.
• Vision Reinforcement Learning (VLM RL): Train Vision/multimodal models via GRPO and RL with Unsloth!
• gpt-oss Reinforcement Learning
• Tutorial: How to Train gpt-oss with RL: Learn to train OpenAI gpt-oss with GRPO to autonomously beat 2048 locally or on Colab.
• Unsloth Dynamic GGUFs on Aider Polyglot: Performance of Unsloth Dynamic GGUFs on Aider Polyglot Benchmarks
• Qwen3-VL: How to Run & Fine-tune: Learn to fine-tune and run Qwen3-VL locally with Unsloth.
• gpt-oss: How to Run & Fine-tune: Run & fine-tune OpenAI's new open-source models!
• Tutorial: How to Fine-tune gpt-oss: Learn step-by-step how to train OpenAI gpt-oss locally with Unsloth.
• Long Context gpt-oss Training
• GLM-4.6: How to Run Locally: A guide on how to run Z.ai's new GLM-4.6 model on your own local device!
• IBM Granite 4.0: How to run IBM Granite-4.0 with Unsloth GGUFs on llama.cpp, Ollama and how to fine-tune!
• DeepSeek-V3.1: How to Run Locally: A guide on how to run DeepSeek-V3.1 and Terminus on your own local device!
• Qwen3-Coder: How to Run Locally: Run Qwen3-Coder-30B-A3B-Instruct and 480B-A35B locally with Unsloth Dynamic quants.
• Gemma 3: How to Run & Fine-tune: How to run Gemma 3 effectively with our GGUFs on llama.cpp, Ollama, Open WebUI and how to fine-tune with Unsloth!
• Gemma 3n: How to Run & Fine-tune: Run Google's new Gemma 3n locally with Dynamic GGUFs on llama.cpp, Ollama, Open WebUI and fine-tune with Unsloth!
• Qwen3: How to Run & Fine-tune: Learn to run & fine-tune Qwen3 locally with Unsloth + our Dynamic 2.0 quants
• Qwen3-2507: Run Qwen3-30B-A3B-2507 and 235B-A22B Thinking and Instruct versions locally on your device!
• Tutorials: How To Fine-tune & Run LLMs: Learn how to run and fine-tune models for optimal performance 100% locally with Unsloth.
• DeepSeek-R1-0528: How to Run Locally: A guide on how to run DeepSeek-R1-0528 including Qwen3 on your own local device!
• Magistral: How to Run & Fine-tune: Meet Magistral - Mistral's new reasoning models.
• Llama 4: How to Run & Fine-tune: How to run Llama 4 locally using our dynamic GGUFs which recovers accuracy compared to standard quantization.
• Kimi K2: How to Run Locally: Guide on running Kimi K2 and Kimi-K2-Instruct-0905 on your own local device!
• Grok 2: Run xAI's Grok 2 model locally!
• Devstral: How to Run & Fine-tune: Run and fine-tune Mistral Devstral 1.1, including Small-2507 and 2505.
• DeepSeek-V3-0324: How to Run Locally: How to run DeepSeek-V3-0324 locally using our dynamic quants which recovers accuracy
• DeepSeek-R1: How to Run Locally: A guide on how you can run our 1.58-bit Dynamic Quants for DeepSeek-R1 using llama.cpp.
• DeepSeek-R1 Dynamic 1.58-bit: See performance comparison tables for Unsloth's Dynamic GGUF Quants vs Standard IMatrix Quants.
• QwQ-32B: How to Run effectively: How to run QwQ-32B effectively with our bug fixes and without endless generations + GGUFs.
• Phi-4 Reasoning: How to Run & Fine-tune: Learn to run & fine-tune Phi-4 reasoning models locally with Unsloth + our Dynamic 2.0 quants
• Running & Saving Models: Learn how to save your finetuned model so you can run it in your favorite inference engine.
• Saving to GGUF: Saving models to 16bit for GGUF so you can use it for Ollama, Jan AI, Open WebUI and more!
• Saving to Ollama
• Saving to vLLM for deployment: Saving models to 16bit for vLLM deployment and serving
• Saving to SGLang for deployment: Saving models to 16bit for SGLang for deployment and serving
• Unsloth Inference: Learn how to run your finetuned model with Unsloth's faster inference.
• Troubleshooting Inference: If you're experiencing issues when running or saving your model.
• vLLM Engine Arguments
• LoRA Hot Swapping Guide
• Text-to-Speech (TTS) Fine-tuning: Learn how to to fine-tune TTS & STT voice models with Unsloth.
• Unsloth Dynamic 2.0 GGUFs: A big new upgrade to our Dynamic Quants!
• Vision Fine-tuning: Learn how to fine-tune vision/multimodal LLMs with Unsloth
• Fine-tuning LLMs with NVIDIA DGX Spark and Unsloth: Tutorial on how to fine-tune and do reinforcement learning (RL) with OpenAI gpt-oss on NVIDIA DGX Spark.
• Fine-tuning LLMs with Blackwell, RTX 50 series & Unsloth: Learn how to fine-tune LLMs on NVIDIA's Blackwell RTX 50 series and B200 GPUs with our step-by-step guide.
• Multi-GPU Training with Unsloth: Learn how to fine-tune LLMs on multiple GPUs and parallelism with Unsloth.
• Finetuning from Last Checkpoint: Checkpointing allows you to save your finetuning progress so you can pause it and then continue.
• Troubleshooting & FAQs: Tips to solve issues, and frequently asked questions.
• Chat Templates: Learn the fundamentals and customization options of chat templates, including Conversational, ChatML, ShareGPT, Alpaca formats, and more!
• Quantization-Aware Training (QAT): Quantize models to 4-bit with Unsloth and PyTorch to recover accuracy.
• Unsloth Environment Flags: Advanced flags which might be useful if you see breaking finetunes, or you want to turn stuff off.
• Continued Pretraining: AKA as Continued Finetuning. Unsloth allows you to continually pretrain so a model can learn a new language.
• Unsloth Benchmarks: Unsloth recorded benchmarks on NVIDIA GPUs.