LoRA

What is LoRA?

Teaching a large model a new skill used to mean full fine-tuning — updating every one of its billions of parameters, which burns compute and money. LoRA (low-rank adaptation) flips the idea: leave the main model untouched and train only a small handful of extra parameters that let it pick up the new task.

How does it pull that off?

Freeze the original model
The existing weights are locked and never updated.
Add a "bypass"
Next to the layers you want to adapt, hang a small "low-rank" module made of two tiny matrices, and train only that bypass.
The low-rank trick
A big matrix update can be approximated by multiplying two small matrices together — orders of magnitude fewer parameters, with results close to full fine-tuning.

Why is it great?

Cheap compute
A normal GPU is enough — no racks of A100s required.
Cheap storage
A LoRA adapter is just tens of megabytes. You can load, switch and stack them like plugins.
Reusable
One base model can wear many LoRAs for different tasks, each doing its own job.

Who is it for?

Teams and individuals who want to customize an open model — a style, a niche domain — without a budget. LoRA turns "fine-tuning a big model" from a big-company luxury into an everyday developer tool.

Bottom line: LoRA moves just a "thin strip" of parameters and leverages the whole large model.