03 · ADAPTER
VerifiedLoRA and QLoRA
Adapt a frozen base with a low-rank update that fits measurable work into the 16 GB class.
LoRA trains only A and B adapters in W' = W + scale × BA. Initializing B at zero starts from the base model's behavior.
QLoRA stores the frozen base in 4-bit form while adapters and critical computation can use higher precision. It expands the feasible space but never guarantees a specific model fits in 16 GB.
01
First thought
QLoRA trains the entire model as 4-bit integers.
02
Correction
The frozen base is 4-bit; small LoRA adapters are trained and compute precision is handled separately.
03
Decision rule
On 16 GB, start with QLoRA; decide quality with an independent benchmark.