--- format: "aidr-story-markdown/v1" id: "23b0ad911bff96ad1bbe4aa54d214fc657102fbda409824727530cfe857a7ca6" canonical_url: "https://aidr.today/23b0ad91?lang=en" title: "Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live iOS Apps" lang: "en" requested_lang: "en" available_langs: ["en","vi"] translation_fallback: null fallback_fields: [] published_at: "2026-11-02T00:00:00.000Z" category: "Research" topics: ["agent"] source_urls: ["https://machinelearning.apple.com/research/planning-grounding-ui-navigation"] summary: "Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live iOS Apps Authors Peitong Duan*, Seyed Khashaiar Gatmiry*, Jeff Nichols, Emmanuel Abbe, Samy Bengio While frontier vision-language models demonstrate strong capabilities in user interface (UI) navigation, their high inference costs prohibit everyday use. Conversely, smaller models trained via supervised fine-tuning (SFT) or offline reinforcement learning often memorize static trajectories, leading to brittle execution and an inability to recover from errors. To address these limitations, we propose a novel online reinforcement learning framework that trains UI navigation agents directly on live iOS applications. Our framework decouples the agent into two specialized components: a Planning agent and a UI Grounding model. We rely on an LLM-as-a-Judge to provide comprehensive reward signals, combining dense, step-level feedback to evaluate intermediate progress with a terminal reward for overall task completion. To effectively leverage this dual reward structure, we optimize the agent using Proximal Policy Optimization (PPO) with step-level advantage estimation. Furthermore, we introduce GiGPPO…" --- # Online Dense Reinforcement Learning with Planning\-Grounding Agent for UI Navigation on Live iOS Apps > [Open the canonical story]() **Published:** 2026-11-02T00:00:00.000Z **Category:** Research **Topics:** agent ## Summary Online Dense Reinforcement Learning with Planning\-Grounding Agent for UI Navigation on Live iOS Apps Authors Peitong Duan\*, Seyed Khashaiar Gatmiry\*, Jeff Nichols, Emmanuel Abbe, Samy Bengio While frontier vision\-language models demonstrate strong capabilities in user interface \(UI\) navigation, their high inference costs prohibit everyday use\. Conversely, smaller models trained via supervised fine\-tuning \(SFT\) or offline reinforcement learning often memorize static trajectories, leading to brittle execution and an inability to recover from errors\. To address these limitations, we propose a novel online reinforcement learning framework that trains UI navigation agents directly on live iOS applications\. Our framework decouples the agent into two specialized components: a Planning agent and a UI Grounding model\. We rely on an LLM\-as\-a\-Judge to provide comprehensive reward signals, combining dense, step\-level feedback to evaluate intermediate progress with a terminal reward for overall task completion\. To effectively leverage this dual reward structure, we optimize the agent using Proximal Policy Optimization \(PPO\) with step\-level advantage estimation\. Furthermore, we introduce GiGPPO… ## Sources - [Story source]()