Vibe Coding Discoverpeft
View on GitHub🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
Hugging Face PEFT implements parameter-efficient fine-tuning methods (LoRA, QLoRA, IA3, prompt tuning, adapters and more) on top of Transformers, Diffusers, Accelerate and TRL. It lets you train and serve large models with a fraction of the GPU memory and storage, with only small adapter checkpoints.
Use Cases
Fine-tune LLMs on consumer GPUs with LoRA/QLoRATrain adapters for Stable Diffusion / DreamBoothServe many downstream tasks from one base model with swappable adaptersReduce checkpoint storage by saving MB-sized adaptersMultilingual ASR fine-tuning with 8-bit quantizationRLHF/DPO fine-tuning of large modelsMerge LoRA adapters into base model weightsDistributed training of very large models with Accelerate
Built With
- Language
- Python
- Frameworks
- PyTorch · Hugging Face Transformers · Hugging Face Diffusers · Hugging Face Accelerate · TRL · bitsandbytes · DeepSpeed · Megatron
Tags
peft · lora · fine-tuning · parameter-efficient · adapters · qlora · soft-prompts · ia3 · diffusers · quantization · llm · pytorch · transformers · training · inference · huggingface