A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Design

arXiv·medium signal

Xie, Ban, and Hong reframe supervised fine-tuning as a target-distribution-design problem rather than uniform per-token likelihood maximization, arguing that not every demonstrated token deserves equal weight. The framework connects existing SFT variants under one lens and could change how agent behaviors are distilled from demonstration trajectories. Useful for anyone fine-tuning models on tool-use or multi-step agent traces.