现代人工智能代理依靠复杂的推理工具（例如 Claude Code, Codex, 和 OpenClaw）来驱动多轮推理, 工具的使用, 和对外部系统的访问。虽然功能强大,，这些复杂的工具也使得代理很难使用开放基础设施,进行端到端训练，而其SFT/RL堆栈无法本机表达有状态的,多进程工具推理。为了解决这个,，我们提出了 OpenForgeRL, 一个开源框架，用于在不同的环境中端到端地训练基于工具的代理。 OpenForgeRL 通过一个轻量级代理来实现这一目标，该代理为harness的 模型调用提供服务，同时将它们记录为标准 RL 代码库 (e.g., verRL), 的训练数据，以及在自己的远程容器, 中运行每个部署的 Kubernetes 编排器，从而能够在任何环境中对任何harness 进行大规模训练。通过将训练和推理, OpenForgeRL 解耦，研究人员可以轻松地训练, 研究,，并直接在部署的真实工具和环境中改进代理。我们在不同的,复杂工具和环境,跨越工具/claw基于代理和多模式GUI浏览器和计算机使用代理中验证我们的框架。仅使用数百到数千个任务, OpenForgeClaw 在 ClawEval 上达到 31.7 pass^3 和 55.9 pass@3 在 QwenClawBench 上达到 33.7。 OpenForgeGUI 在 OSWorld-Verified, 上达到 37.7，在 Online-Mind2Web, 上达到 63.0，在 WebVoyager 上达到 72.3。两者在几乎所有基准, 上都优于类似大小的开放基线，并且在 GUI 设置中匹配或超过数倍大的模型。除了基准,之外，我们还分析了如何利用选择(例如, ZeroClaw, OpenClaw, Codex)和RL形状代理行为。我们发现，某些工具比其他工具更难学习,，并且 RL 提高了代理可靠性,，例如自我验证, 工具覆盖率, 和完成多步骤计划,，尽管错误恢复等关键能力仍然很弱。

Modern AI agents rely on elaborate inference harnesses such as Claude Code, Codex, and OpenClaw to drive multi-turn reasoning, tool use, and access to external systems. While powerful, these complex harnesses also make agents hard to train end-to-end with open infrastructure, whose SFT/RL stacks cannot natively express stateful, multi-process harness inference. To address this, we present OpenForgeRL, an open-source framework for training harness-based agents end-to-end in diverse environments. OpenForgeRL achieves this with a lightweight proxy that serves the harness的 model calls while recording them as training data for a standard RL codebase (e.g., veRL), and a Kubernetes orchestrator that runs each rollout in its own remote container, together enabling training on any harness in any environment at scale. By decoupling training and inference, OpenForgeRL allows researchers to easily train, study, and improve agents directly in the real harnesses and environments they are deployed with. We validate our framework across diverse, complex harnesses and environments, spanning tool/claw-based agents and multimodal GUI browser- and computer-use agents. Using only hundreds to a few thousand tasks, OpenForgeClaw reaches 31.7 pass^3 and 55.9 pass@3 on ClawEval and 33.7 on QwenClawBench. OpenForgeGUI reaches 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager. Both outperform open baselines of similar size on nearly all benchmarks, and in the GUI setting match or surpass models several times larger. Beyond benchmarks, we analyze how harness choice (e.g., ZeroClaw, OpenClaw, Codex) and RL shape agent behavior. We find that some harnesses are substantially harder to learn than others, and that RL improves agentic reliability, such as self-verification, tool coverage, and completing multi-step plans, though critical abilities such as error recovery remain weak.

科目: 人工智能 (cs.AI); 计算和语言 (cs.CL)

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)