Frozen-reference anchoring
Frozen-reference anchoring uses a separately loaded non-trainable model as the reference for Trainfer preference and KL objectives.
Project status: Implemented and opt-in through cfg.frozen_ref. This entry describes the source audit of 14 September 2026; historical measurements retain their original dates.
Mechanism
An external reference avoids the drift of the inexpensive adapter-disabled live-model reference after residual consolidation. Objectives compare against fixed reference outputs while the active model continues training.
Implementation and controls
ModelState.load_frozen_ref loads a second base model from base_model_name, sets eval mode, disables gradients, and uses 4-bit loading. TrainEngine passes it as pi_ref where no explicit reference was supplied. The default is false to avoid the second model's memory cost.
Evidence and evaluation
The optimizer and forgetting synthesis identifies reference drift as a concern, and the implementation provides a stable-base alternative. This is infrastructure supporting experiments; an isolated frozen-ref retention win is not established by the inspected reports.
Limitations and interpretation
The loader constructs the named base model: it does not clone an arbitrary currently adapted “best snapshot.” The code's “session-start policy” wording is only exact when that start equals the loaded base. Distinguish this from snapshot self-distillation. A fixed reference may resist learning legitimately new facts, and its VRAM cost competes with rollout capacity.
Sources
- trainfer: trainfer/state.py — checkout audited
1c6391f3773b. - trainfer: trainfer/engine/train.py — checkout audited
1c6391f3773b. - trainfer: trainfer/config.py — checkout audited
1c6391f3773b. - cont: docs/research/sample-efficiency-synthesis.md — checkout audited
87946914c7b9.