Recovering Reliable Supervision via Critique-Guided On-Policy Distillation

A new method called CriticOPD addresses failures in on-policy distillation when teachers follow student errors instead of correcting them. The approach uses teacher critiques of incorrect rollouts to guide student revisions, restoring the teacher's recoverability. Experiments on two Qwen3 students and six mathematical reasoning benchmarks show CriticOPD improves on-policy distillation, with the largest gains on competition problems.