LLM 编码代理通过接口发出 Bash 命令，这些命令可以序列化, 包装, 并重新解析模型输出。仅匹配的执行分数无法区分命令生成错误和生成后引入的故障。 QuoteBench 通过对来自 14 个事件派生系列, 的 56 个一次性任务进行精确的最终状态验证来衡量这一边界，这些任务跨越了生成合同，并围绕一个故意未转义的附加解析器执行传输。在插值点处进行转义会重现每个重播的回复的 原始路径结果,，因此公开边界下的任何恢复都必须来自更改其生成的模型。在八个相同窗口配置, 中，通过添加的解析器重播相同的回复会将成功率降低 55.4 到 73.2 个百分点; 披露对于六种配置, 来说恢复了 30.4 到 60.7 个百分点，而其他两个配置则为零或略有负数。原始生成在前沿几乎饱和; 边界适应仍然是模型之间的区别。 GPT-5.6-sol的 匹配的 -3.6 点差距隐藏了 -64.3 点伤害和 +60.7 点补偿。部署配置重新排序模型: 26 个可比对中的一个逆转是明确的，另外四个位于单任务边缘。命令发布代理的评估应报告模型配置,生成合约,执行路径,操作点,和最终状态验证器，而不是将匹配分数视为内在模型属性。

LLM coding agents issue Bash commands through interfaces that may serialize, wrap, and reparse model output. Matched execution scores alone cannot distinguish command-generation errors from failures introduced after generation. QuoteBench measures this boundary with exact final-state validation on 56 one-shot tasks from 14 incident-derived families, crossing the generation contract with the execution transport around one deliberately unescaped added parser. Escaping at the interpolation point reproduces each replayed reply的 raw-path outcome, so any recovery under a disclosed boundary must come from the model changing its generation. Across eight same-window configurations, replaying the same reply through the added parser lowers success by 55.4 to 73.2 percentage points; disclosure recovers 30.4 to 60.7 points for six configurations, and zero or slightly negative for the other two. Raw generation is nearly saturated at the frontier; boundary adaptation is what still separates models. GPT-5.6-sol的 matched gap of -3.6 points hides -64.3 points of damage and +60.7 points of compensation. The deployment configuration reorders models: one reversal among 26 comparable pairs is unambiguous and four more sit on single-task margins. Evaluations of command-issuing agents should report the model configuration, generation contract, execution path, operating point, and final-state validator rather than treat a matched score as an intrinsic model property.

科目: 人工智能 (cs.AI); 软件工程 (cs.SE)

Subjects: Artificial Intelligence (cs.AI); Software Engineering (cs.SE)