图形用户界面 (GUI) 基础将自然语言指令映射到目标元素的视觉坐标，并作为自主 GUI 代理的核心功能。最近的强化学习方法(例如, GRPO)已经取得了强大的性能,，但它们依赖于昂贵的多次部署，并且在硬样本上受到稀疏信号的影响。这些限制使得政策自蒸馏 (OPSD), 成为一种有希望的替代方案，它通过一次部署即可提供密集的代币级监督,。然而, 它对 GUI 基础的适用性仍未得到探索。在本文, 中，我们提出了 GUI-SD, 第一个专为 GUI 基础而定制的 OPSD 框架。首先, 它使用目标边界框和高斯软掩模, 为教师构建视觉丰富的特权上下文, 提供信息指导而不会泄漏精确坐标。其次,它采用熵引导蒸馏,，根据数字重要性和教师置信度,自适应地对标记进行加权,将优化集中在最具影响力和最可靠的位置上。对六个代表性 GUI 基础基准的广泛实验表明，GUI-SD 在准确性和训练效率方面始终优于基于 GRPO 的方法和朴素 OPSD。代码和训练数据可从此 https URL 获取。
Graphical User Interface (GUI) grounding maps natural language instructions to the visual coordinates of target elements and serves as a core capability for autonomous GUI agents. Recent reinforcement learning methods (e.g., GRPO) have achieved strong performance, but they rely on expensive multiple rollouts and suffer from sparse signals on hard samples. These limitations make on-policy self-distillation (OPSD), which provides dense token-level supervision from a single rollout, a promising alternative. However, its applicability to GUI grounding remains unexplored. In this paper, we present GUI-SD, the first OPSD framework tailored for GUI grounding. First, it constructs a visually enriched privileged context for the teacher using a target bounding box and a Gaussian soft mask, providing informative guidance without leaking exact coordinates. Second, it employs entropy-guided distillation, which adaptively weights tokens based on digit significance and teacher confidence, concentrating optimization on the most impactful and reliable positions. Extensive experiments on six representative GUI grounding benchmarks show that GUI-SD consistently outperforms GRPO-based methods and naive OPSD in both accuracy and training efficiency. Code and training data are available at this https URL.
科目: 人工智能 (cs.AI); 计算机视觉和模式识别 (cs.CV)
Subjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)