命令行界面 (CLI) 代理正在成为代理与计算机交互的实用范例，通过不断发展的文件系统, 可执行命令行程序, 和在线执行反馈。最近的工作使用强化学习(RL) 从可验证的任务反馈, 中学习这些交互能力，但很少有方法利用 CLI 操作的本机结构化属性作为学习信号。除了这种未充分利用的操作结构之外，, CLI 学习还给编码代理带来了两个瓶颈。首先, 代理必须从部分观察中识别大型代码库中与任务相关的证据。 Second, 稀疏终端奖励必须分配给形成长多回合轨迹的动作。我们通过 shell 驱动的信息提取和文件编辑任务来研究这些瓶颈。对于选择性观察,，我们引入了$\sigma$-Reveal, 推理时间机制，为同一 CLI 选择令牌预算上下文。对于信用分配,，我们建议行动优势分配($\mathrm{A}^3$),，这是一种原生代理强化学习方法，保留了标准代理强化学习的算法复杂性。 $\mathrm{A}^3$ 从情节级相对反馈,抽象语法树(AST)基于动作子链残差,和树级轨迹边际构造回合级优势。为了进一步评估这个问题设置,，我们构建了 ShellOps, 一个可验证的数据集套件，涵盖存储库环境中的 CLI 任务。
Command line interface (CLI) agents are emerging as a practical paradigm for agent-computer interaction over evolving filesystems, executable command line programs, and online execution feedback. Recent work has used reinforcement learning (RL) to learn these interaction abilities from verifiable task feedback, yet few methods exploit the native structured attributes of CLI actions as learning signals. Beyond this underused action structure, CLI learning also couples two bottlenecks for coding agents. First, the agent must identify task-relevant evidence in a large codebase from partial observations. Second, sparse terminal rewards must be assigned to the actions that shape a long multi-turn trajectory. We study these bottlenecks through shell-driven information extraction and file editing tasks. For selective observation, we introduce $\sigma$-Reveal, an inference-time mechanism that selects token-budgeted context for the same CLI. For credit assignment, we propose Action Advantage Assignment ($\mathrm{A}^3$), a native agentic RL method that preserves the algorithmic complexity of standard agentic RL. $\mathrm{A}^3$ constructs turn-level advantages from episode-level relative feedback, abstract syntax tree (AST) based action sub-chain residuals, and tree-level trajectory margins. To further evaluate this problem setting, we construct ShellOps, a verifiable dataset suite covering CLI tasks in repository environments.
科目:人工智能(cs.AI)
Subjects: Artificial Intelligence (cs.AI)