Focused Omni-Modal GUI Agents

Researchers propose Focused Perception as a step-adaptive, action-sufficient, and non-intrusive representation for GUI agents operating over streaming omni-modal observations. They introduce FocusGUI, a lightweight adapter that dynamically prioritizes multimodal channels against a step-specific decision reference and injects compressed evidence into a frozen decoder's embeddings without altering its native parameter space. FocusGUI trains from action supervision alone, requiring neither modality-relevance labels nor backbone fine-tuning.