计算机使用代理在 OSWorld 和 AndroidWorld, 等基准上取得了进步，但仍然主要通过 GUI, 进行操作，通常会产生低效的轨迹。现实世界的计算机工作是混合,，将视觉状态检查与精确, 高吞吐量命令行操作, 相结合，因此有能力的代理必须在共享应用程序状态上协调这两种模式。然而，可扩展的混合环境仍然稀缺，因为在实际应用程序上同时支持 GUI 和 CLI 通常需要为每个应用程序进行大量的手动工程。现有代理也很难互补地使用这两个界面: CLI 本机代理缺乏对涉及界面状态或布局的任务的视觉感知,，而 GUI 本机代理对于通过命令更好地执行的操作效率低下。我们引入了 CUA-Universe,，这是一个可扩展的环境到数据管道，可将真正的桌面软件转变为混合 GUI+CLI 环境。 App-Forge 将应用程序调整为可复制的虚拟机和命令行界面，它发现, 包装, 或生成, 扩展到 16 个应用程序; Task-Weave 通过对种子文件的可重用操作来合成可控难度的各种混合任务; 和 Path-Steer 沿着高效的混合路径引导部署，并收获经过验证的轨迹以进行后期训练。对此数据的训练将行为从低效的 GUI 交互和脆弱的 CLI 脚本转变为有效的 GUI+CLI 编排。我们的 9B 模型提高了 CUA-Verse (Score +39.3 pts; -37%steps, -60% tokens), OSWorld (SR +16.8 pts; -57%steps, -44% tokens), 和 OSWorld-MCP (Score 的成功率和效率+7.84 分; -27% 步, -30% token)。 CUA-Universe 提供了一条可扩展的途径，使计算机使用代理的能力更强、更高效。
Computer-use agents have advanced on benchmarks like OSWorld and AndroidWorld, but still act mostly through the GUI, often producing inefficient trajectories. Real-world computer work is hybrid, combining visual-state inspection with precise, high-throughput command-line operations, so capable agents must coordinate both modalities over shared application state. Yet scalable hybrid environments remain scarce because supporting both GUI and CLI over real applications typically requires substantial manual engineering for each application. Existing agents also struggle to use the two interfaces complementarily: CLI-native agents lack visual perception for tasks involving interface state or layout, while GUI-native agents are inefficient for operations better executed through commands. We introduce CUA-Universe, a scalable environment-to-data pipeline that turns real desktop software into hybrid GUI+CLI environments. App-Forge adapts applications into reproducible VMs and command-line surfaces it discovers, wraps, or generates, scaling to 16 applications; Task-Weave synthesizes diverse hybrid tasks of controllable difficulty from reusable operations over seed files; and Path-Steer steers rollouts along efficient hybrid paths and harvests verified trajectories for post-training. Training on this data shifts behavior from inefficient GUI interaction and brittle CLI scripting toward effective GUI+CLI orchestration. Our 9B model improves both success and efficiency on CUA-Verse (Score +39.3 pts; -37% steps, -60% tokens), OSWorld (SR +16.8 pts; -57% steps, -44% tokens), and OSWorld-MCP (Score +7.84 pts; -27% steps, -30% tokens). CUA-Universe provides a scalable path toward more capable and efficient computer-use agents.
科目:人工智能(cs.AI)
Subjects: Artificial Intelligence (cs.AI)