Low-Rank Adaptation

Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning (PEFT) technique that freezes pre-trained backbone weights and injects trainable low-rank decomposition matrices into targeted linear layers. This drastically reduces GPU memory overhead and enables hot-swapping multiple lightweight task-specific adapters on a shared base model instance.