Entity-masked SFT
Entity-masked SFT is an answer-suffix-only training recipe intended to avoid spending direct supervision on a repeated answer prefix.
Project status: Implemented as entity_masked_sft. This entry describes the source audit of 14 September 2026; historical measurements retain their original dates.
Mechanism
Build the target Answer: <expected> and pass span_prefix="Answer: " so weighted SFT supervises only the trailing answer span. Repeat a fixed number of passes per task instead of using the memorize loop's rank-based early stopping.
Implementation and controls
The runner uses the first N training tasks, defaults in its branch to five tasks and 30 passes, and reads overrides from the config (currently 60 passes and sample weight 1). It submits one sample per update. This is prefix masking applied to final-answer supervision, not a named-entity recognition model or entity-specific parameter partition.
Evidence and evaluation
The May 16 journal reports heldout 0.50 and probe 0.93 at both 30 and 60 passes, versus memorize's 0.60 heldout in that comparison. Probe gains therefore came with a weaker forward score. The small probe overlaps the training grammar and does not establish broad preservation.
Limitations and interpretation
Fixed passes and early-stopped memorize have different update budgets. Suffix-only loss still updates shared parameters. The sample weight cancels in singleton normalized weighted SFT; changing that weight alone is not a reliable way to change update magnitude.
Sources
- cont: autoresearch/experiment.py — checkout audited
87946914c7b9. - cont: autoresearch/config.json — checkout audited
87946914c7b9. - cont: docs/research/JOURNAL.md — checkout audited
87946914c7b9. - trainfer: trainfer/objectives/sft.py — checkout audited
1c6391f3773b.