CrEST: let the teacher set the magnitude of the update, not the direction, for multi-turn tool-use agents

arXiv·medium signalXBlueskyLinkedInCopy link

RLVR gives multi-turn tool-use agents a verifier-bounded ceiling but collapses heterogeneous per-turn outcomes into one trajectory-level reward, while on-policy distillation gives dense per-token signal but is teacher-bounded or collapses under gradient concentration. CrEST (arXiv 2608.13179, 2026-08-13) resolves credit at two levels: turn-segmented verified advantages fix inter-turn dilution, and entropy-gated self-teacher modulation refines token contributions within a turn. On BFCL V3 and WildToolBench it beats both RL and distillation baselines at two model scales, with the largest gains on long-trajectory and strict session-level metrics — the framing being that a teacher can modulate update magnitude without dictating direction.