PEFT is orthogonal — LoRA · QLoRA · DoRA

Every method chapter so far — imitation, hint-guided bootstrapping, preference, reinforcement, agentic and long-horizon RL — has been about what signal moves the weights. This closing chapter of the methods survey is about something orthogonal: how much of the weight matrix is even allowed to move, and why that’s a separate decision from the method itself.

Common confusion worth killing outright: PEFT is not a fine-tuning method — it’s a mechanism for applying one. Any of SFT / DPO / GRPO / RLVR can be delivered full-parameter or via a PEFT adapter. It changes which parameters get gradients and how much memory you burn, not what signal you learn from.

The methods

• LoRA — freeze W, train a low-rank update ΔW = B·A (rank r), so y = Wx + (BA)x·(α/r). Only A, B get gradients (Hu et al., arXiv:2106.09685).
• QLoRA — quantize the frozen base to 4-bit NF4, keep adapters in BF16; lets a large base fit a small GPU (Dettmers et al., arXiv:2305.14314).
• DoRA — decompose the update into magnitude + direction for a bit more accuracy at the same budget (arXiv:2402.09353).

Engineering facts that matter

• It’s a knob on top of a method. “Should I do LoRA or GRPO?” is a category error — you do GRPO, via LoRA. On-policy distillation reproductions run rank-128 LoRA; OpenAI/Fireworks/Google Vertex customer-RFT products are LoRA-first — LoRA lives in the applied/enterprise fine-tuning layer. The frontier labs post-train their own flagship checkpoints full-parameter (Llama/Qwen/DeepSeek/GPT/Claude/Gemini reports; verified 2026 pass).
• LoRA bounds drift magnitude, not behavioral direction — it is not an unconditional forgetting guarantee. The low-rank cap does measurably protect broad, high-rank capability (held-out benchmarks stay closer to base than full-FT’s, across code/math — Biderman et al., “LoRA Learns Less and Forgets Less,” TMLR 2025, arXiv:2405.09673). But format/policy switches (e.g. “always emit <think>”, “always refuse X”) are themselves low-rank — a rank-8–64 adapter has ample capacity to flip them if the training signal is consistent, and LoRA-trained weight matrices develop “intruder dimensions” with no analog in full-FT that compound across sequential adapter rounds (Shuttleworth et al., arXiv:2410.21228; safety-refusal collapse case study, Lermen et al., arXiv:2310.20624). Full protect-vs-doesn’t-protect table, the rank/α/LR knobs that actually help, and the CoT-collapse-under-LoRA case study relevant to the small-model overwrite problem in Imitation: Data mixing, ratios & not forgetting how to think §2.
• It is startlingly learning-rate-sensitive — more so than which adapter variant you pick. The 2026 unified LoRA-variant study finds a well-tuned vanilla LoRA matches or beats most fancy variants, and LoRA needs a higher LR than full-FT to learn comparably (recommended range 5e-5 to 5e-4) — a double-edged knob, since cranking LR to make LoRA learn also erodes the forgetting protection above (arXiv:2601.22708; LR-sensitivity finding corroborated by Biderman et al., arXiv:2405.09673 and Thinking Machines, “LoRA Without Regret” (2025)). Tune LR before you tune adapter architecture. Two lower-profile knobs worth knowing: rsLoRA rescales by α/√r instead of α/r to stop high-rank configs from destabilizing (use_rslora in HF PEFT, Kalajdzievski, arXiv:2312.03732); DoRA and rsLoRA are learning-capacity accelerants (close the LoRA-vs-FFT gap faster/further), not forgetting fixes.

Practical default for your scale

At ≤~9–16B, LoRA/QLoRA is the sane default for iteration cost; go full-parameter only when you have a concrete reason (measured OOM headroom aside, the project’s stance is LoRA-by-default, full-FT as a deliberate escalation — see lessons/post-training/ in shared memory). It composes with every method chapter here — the same on/off-policy axis and fixed method presets apply whether the gradients land in the full weight matrix or a rank-r adapter (Foundations: the one axis).

One stage is the recipe’s own exception to “LoRA-by-default”: continued/domain pretraining needs to learn too much (new facts, new token distributions) for a low-rank constraint to absorb, so full fine-tuning, not LoRA, is the recommended method there — full-FT learns perturbations at 10–100× the effective rank of typical LoRA configs (The recipe is a sequence, not a pick, stage 1). On-policy distillation reproductions running rank-128 LoRA are a T3 “promising, watch” entry, not yet a flagship-proven default — see the full tier ranking in Start here: a proven-first ranking.