Fine-tuning a 70B model used to mean a data center and $4,000. Then LoRA cut trainable params by 99% by observing that fine-tuning updates are intrinsically low-rank. Then QLoRA pushed it to 4-bit and squeezed the same workload onto a single consumer GPU. By 2026, Hugging Face hosts 50,000+ LoRA adapters and you can fine-tune Llama 3.3 70B on a gaming laptop for the price of a coffee.
Learning Objectives
After this lesson, you will be able to:
Know when to fine-tune vs. just write a better prompt -- and have a clear decision framework
Understand how LoRA decomposes weight updates into low-rank matrices B and A, and why this captures most of the fine-tuning signal
See how QLoRA combines 4-bit NormalFloat quantization with LoRA to enable fine-tuning a 7B model on a single consumer GPU
Walk through the complete SFT pipeline: dataset curation, chat-template formatting, loss masking on instruction tokens, and evaluation
Compare parameter-efficient methods: LoRA vs adapter layers vs prefix tuning vs IA3 -- and know when each is the right choice
Understand catastrophic forgetting and why freezing base weights in LoRA prevents it
You have 1,000 customer service conversations from your company. You want an AI that handles customer queries in your company's tone and knows your product catalog. Fine-tune or prompt engineer?
Use prompt engineering when: You need format changes, simple instruction following, or have fewer than 100 examples.
Use RAG when: You need factual accuracy, up-to-date information, or your knowledge base changes frequently.
Use fine-tuning when: You need consistent tone/style, domain-specific reasoning, or the model must internalize patterns too complex for a prompt.
Combine them: The best production systems often use all three. Fine-tune for domain expertise and tone, RAG for factual grounding, and prompt engineering for per-request formatting.
Full fine-tuning updates every parameter in the model. For a 7B parameter model, this means:
Memory: ~28 GB just for the model weights in fp32, plus ~28 GB for gradients, plus ~56 GB for optimizer states (Adam stores two states per parameter). Total: ~112 GB. You need multiple A100 GPUs.
Cost: Hundreds to thousands of dollars per training run.
Risk: Catastrophic forgetting -- the model loses its general capabilities as it overfits to your narrow task. A fine-tuned medical model might forget how to write code.
LoRA (Low-Rank Adaptation of Large Language Models) is the breakthrough that made fine-tuning practical. The key insight: weight updates during fine-tuning live in a low-dimensional subspace.
During fine-tuning, each weight matrix W in the model gets updated to W + delta_W. LoRA observes that delta_W has low rank -- it can be decomposed into the product of two much smaller matrices:
ΔW=B⋅AwhereW∈Rd×k,B∈Rd×r,A∈Rr×k,r≪min(d,k)
Where r is the rank -- typically 4, 8, 16, or 32. This is dramatically smaller than the full weight matrix.
Try it: LoRA decomposition — W + B·A with adjustable rankInteractive
Loading visualization...
Explore this: Drag the rank slider from 1 to 32 and watch how the approximation quality changes. Notice how even rank 4 captures most of the important structure? That is why LoRA works — you only need a few dimensions to capture the changes needed for fine-tuning. A weight matrix W (1000×1000 = 1M parameters) gets decomposed into two small matrices A (1000×4) and B (4×1000) — just 8,000 parameters total.
⚡ Playground:Embeddings → — explore how fine-tuning shifts word representations in embedding space, moving them toward the target domain.
Try it: Visualize model compression with LoRAInteractive
The adapter is tiny -- you can share it on a USB drive. And you can merge it back into the base model for zero-cost inference, or swap between multiple adapters for different tasks.
4-bit NormalFloat quantization: Compresses the base model weights from 16-bit to 4-bit, reducing memory by 4x while preserving quality through a distribution-aware quantization scheme
Double quantization: Quantizes the quantization constants themselves, saving an additional ~0.4 bits per parameter
Paged optimizers: Uses CPU RAM as overflow when GPU memory is insufficient, with intelligent paging to minimize performance impact
The result: a 65B parameter model that would normally require 130 GB of GPU memory (in fp16) fits in ~33 GB with QLoRA. A 7B model fits in ~5 GB, easily running on an RTX 3060.
Insert small bottleneck layers between existing transformer layers. The adapter compresses the hidden state to a smaller dimension, applies a nonlinearity, then projects back up. Only the adapter parameters are trained.
Pros: Clean separation between base model and adaptation. Cons: Adds latency (extra layers in the forward pass).
Prepend learnable "virtual tokens" to the input at every layer. These virtual tokens are not real words -- they are continuous vectors that the model learns to attend to, effectively steering its behavior.
Pros: Very few parameters. Cons: Consumes part of the context window, can be harder to tune.
#IA3 (Infused Adapter by Inhibiting and Amplifying Inner Activations)
Learn three vectors that rescale the keys, values, and feed-forward activations. Even fewer parameters than LoRA.
Pros: Minimal parameter count. Cons: Less expressive than LoRA for complex adaptations.
Your dataset consists of instruction-response pairs:
{"instruction": "Summarize this customer complaint in one sentence.",
"input": "I ordered a blue shirt but received a red one...",
"output": "Customer received wrong shirt color (red instead of blue)."}
Quality matters more than quantity. 1,000 high-quality, diverse examples often outperform 100,000 noisy ones. Key principles:
Diverse instructions: Cover the range of tasks you want the model to handle
Consistent format: Use the same instruction template throughout
Clean outputs: The model will replicate any errors in your training outputs
Deduplication: Remove near-duplicates that would cause memorization
Convert your data into the model's chat template format. Each model family has its own template:
<|im_start|>system
You are a helpful customer service agent.<|im_end|>
<|im_start|>user
Summarize this complaint: I ordered a blue shirt...<|im_end|>
<|im_start|>assistant
Customer received wrong shirt color.<|im_end|>
Critical: mask the loss on instruction tokens. The model should only be trained to predict the response, not the instruction. Otherwise it wastes capacity learning to predict your prompts.
Tests · Verify LoRA r=8 uses less than 1% of full fine-tuning parameters. Verify QLoRA 4-bit uses less than half the memory of fp16.
LoRA: Low-Rank Adaptation of Large Language Models
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen (2021)
The paper that democratized LLM fine-tuning. Shows that weight updates during fine-tuning have low intrinsic rank, enabling decomposition into much smaller matrices. Achieves comparable performance to full fine-tuning with 10,000x fewer trainable parameters on GPT-3 175B.
QLoRA: Efficient Finetuning of Quantized Language Models
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer (2023)
Combines 4-bit NormalFloat quantization with LoRA to enable fine-tuning a 65B model on a single 48GB GPU. Introduced the Guanaco family of models, which matched ChatGPT on the Vicuna benchmark despite being trained on a single GPU in 24 hours.
Fine-tune when prompt engineering and RAG are not enough -- when you need deep behavioral changes, consistent domain expertise, or your customization cannot fit in a context window, fine-tuning is the answer
LoRA decomposes weight updates into low-rank matrices -- instead of updating all 7B parameters, train two small matrices B and A where delta_W = B * A, reducing trainable parameters to 0.1-1% while matching full fine-tuning performance
QLoRA enables consumer-GPU fine-tuning -- 4-bit quantization of the frozen base model plus fp16 LoRA adapters lets you fine-tune a 7B model on a 12 GB GPU and a 70B model on a single A100
The SFT pipeline is dataset -> tokenize -> train -> evaluate -- quality of training data matters more than quantity; always mask loss on instruction tokens; evaluate with held-out data, not just training loss
LoRA adapters are tiny and swappable -- a 16 MB adapter file can specialize a 14 GB base model; serve one base model with multiple adapters for different customers or tasks
A 4096x4096 weight matrix has 16.8M parameters. With LoRA rank r=8, how many parameters does the adapter have?
Next up: RLHF & DPO -- how the same fine-tuning machinery gets pointed at human preferences instead of labelled examples, turning a raw pretrained model into one that follows instructions.