Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation

Waypoint-OPD introduces an intermediate teacher checkpoint to compare early and late shifts in relative action preferences at student-visited prefixes, gating the endpoint shift based on directional agreement between early and late shifts. The method suppresses endpoint shifts when early and late shifts point in opposing directions, requiring only one additional frozen-teacher scoring pass compared to endpoint-only transfer. Evaluations use RL-trained Qwen3-4B teachers with Qwen3-8B, Qwen3-14B, and Qwen3-30B-A3B student models in mathematical and code reasoning tasks.