After this lesson, you will be able to: Explain how LoRA injects low-rank adapters into a model, what QLoRA adds with 4-bit quantization, and run a LoRA fine-tune with the PEFT library.
LoRA is the method that put fine-tuning within reach of students and small labs. This lesson covers the idea (low-rank updates), the QLoRA extension (4-bit base), and a runnable fine-tune you can do on a free GPU.
LoRA (Low-Rank Adaptation) assumes the weight change needed to adapt a model is low-rank. Instead of updating a big weight matrix W directly, it freezes W and learns two small matrices A and B whose product (rank r, e.g. 8 or 16) is added to W: the effective weight is W + BA. You train only A and B, which are tiny compared to W. At inference you can merge BA back into W for zero added latency, or keep adapters separate to swap tasks. The key hyperparameters are the rank r and a scaling factor alpha, plus which layers to adapt (usually the attention projections).
QLoRA loads the frozen base model in 4-bit precision (NF4 quantization) and trains LoRA adapters on top in 16-bit. Because the huge frozen base is now 4x smaller in memory and only the adapters carry gradients, you can fine-tune a 7B (even 13B) model on a single consumer GPU. QLoRA added a few tricks (NF4 data type, double quantization, paged optimizers) and showed near-full-fine-tune quality. It is the default recipe for academic-budget LLM fine-tuning today.
The core of a real fine-tuning script; runs on a free Colab GPU for a small model.
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import LoraConfig, get_peft_modelmodel_id = "meta-llama/Llama-3.2-1B-Instruct"tok = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id) # add load_in_4bit=True for QLoRAlora = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05,target_modules=["q_proj", "k_proj", "v_proj", "o_proj"], # adapt attentiontask_type="CAUSAL_LM",)model = get_peft_model(model, lora)model.print_trainable_parameters()# -> trainable params: ~3M || all params: ~1.2B || trainable%: ~0.3%# Then train with the HF Trainer / SFTTrainer on your instruction dataset,# and model.save_pretrained('my-adapter') saves ONLY the small adapter.
Setting rank too high (wasting memory) or too low (underfitting) without trying a couple of values. Adapting only some layers and not reporting which target_modules you used (it materially affects results and reproducibility). Forgetting that the LoRA scaling is alpha/r, so changing r without alpha changes the effective update size. Evaluating the base model and the adapted model with different prompts or chat templates, which confounds the comparison. Shipping the merged model without noting it was a LoRA fine-tune, which hides the method from anyone reproducing you.
Pick the correct answer.
Sign in and purchase access to unlock this lesson.