RFT research variants

From The Hei Canon

RFT research variants records additional rejection-fine-tuning relatives surveyed in the historical agi research branch.

Project status: Literature/design candidates, distinct from the measured local RFT, STaR, KTO and V-KTO arms. This entry describes the source audit of 14 September 2026; historical measurements retain their original dates.

Mechanism

The historical note separates:

  • RIFT (signed-weighted): use both positive and negative verifier information through signed weighting rather than train only winners.
  • Prefix-RFT (demo-anchored): supply a demonstration prefix during candidate generation, intended to increase useful positive trajectories.
  • STARS (block rejection): reject/resample blocks at inference time; the project labels it outside the training-arm scope.

The same note motivates AdaSTaR-RFT and Hint-RFT, which have their own entries.

Implementation and controls

RFT_LITERATURE_NOTES.md ranks these against the campaign's sparse-positive and surface-form-commit failure modes. The inspected historical runner offers only rft, star, and kto; no RIFT, Prefix-RFT, or STARS mode was identified. Proposed adaptation must specify reward signs, masks, prefix removal, or inference rejection budgets as appropriate.

Evidence and evaluation

The note is a literature synthesis following negative local results. It does not record completed local tests of these three variants. “Out of scope” for STARS means a training comparison should not silently count its additional inference compute as learned improvement.

Limitations and interpretation

Signed negative weights have different stability properties from ordinary positive-only SFT. Prefix conditioning can leak target information; block rejection pays inference cost and does not persist facts into weights. Avoid treating these names as aliases for the current V-KTO implementation.

Sources

See also

Retrieved from "/index.php?title=RFT_research_variants&oldid=1032"