When Does External Guidance Help LLM Reasoning? A Bias-Variance Theory of Guidance-Augmented GRPO

Researchers proposed Guidance-Augmented GRPO (GA-GRPO), a theoretical framework that casts external guidance as a stochastic operator rewriting the question distribution and analyzes the resulting policy-gradient estimator as a biased on-policy estimator whose bias is bounded by the total-variation guidance divergence. The framework subsumes vanilla GRPO, LUFFY, ExPO, PAPO, and TAPO as special cases.