长视野搜索代理必须执行多个连续操作 (steps) 来搜索, 检索, 验证, 并整合证据以获得最终答案。然而，, 训练这些智能体的现有方法通常在监督微调 (SFT) 和强化学习 (RL), 期间统一处理轨迹内的所有步骤，无法区分有用的动作和错误或冗余的动作。在本文,中，我们提出了答案回溯信用分配(ABC),，这是一种细粒度的信用分配框架，用于通过将稀疏轨迹级结果转换为密集的步骤级监督来训练长视野搜索代理，该监督奖励有用的操作(，即使在失败的轨迹)中，同时抑制错误或冗余的操作。具体来说，, 给定一个可能晦涩难懂的查询及其相应的真实答案, ABC 首先执行答案回溯线索恢复,，它从答案中回溯以恢复解决问题所需的中间线索。然后，它应用线索锚定步骤评分来根据这些线索评估每个搜索步骤,，将稀疏的二元结果监督转换为密集的步骤级奖励。基于这些奖励,，我们开发了 ABC-SFT,，它重新加权每回合, 的损失，以及 ABC-GRPO,，它使用步骤级别分数作为 GRPO 中的奖励。在此框架, 的基础上，我们仅使用 8.5k 个示例来基于 Qwen3.5-4B 训练 ABSeeker。 ABSeeker 在 BrowseComp 上获得 37.3%，在 BrowseComp-ZH 上获得 39.1%。通过上下文管理,，分数进一步分别提高到 55.3% 和 52.9%,，显着优于同规模的 (4B) 代理，甚至与较大代理 ( 的性能约 30B) 相当。这些结果证明了答案回溯步骤级信用分配对于训练长视野搜索代理的有效性。
Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer. However, existing methods for training these agents typically treat all steps within a trajectory uniformly during both supervised fine-tuning (SFT) and reinforcement learning (RL), failing to distinguish useful actions from erroneous or redundant ones. In this paper, we propose Answer-Backtracked Credit Assignment (ABC), a fine-grained credit assignment framework for training long-horizon search agents by converting sparse trajectory-level outcomes into dense step-level supervision that rewards useful actions (even in failed trajectories) while suppressing erroneous or redundant actions. Specifically, given a potentially obscure query and its corresponding ground-truth answer, ABC first performs Answer-Backtracked Clue Recovery, which traces back from the answer to recover intermediate clues required to solve the question. It then applies Clue-Anchored Step Scoring to evaluate each search step against these clues, converting sparse binary outcome supervision into dense step-level rewards. Based on these rewards, we develop ABC-SFT, which reweights the loss of each turn, and ABC-GRPO, which uses the step-level scores as rewards in GRPO. Building on this framework, we train ABSeeker based on Qwen3.5-4B with only 8.5k examples. ABSeeker achieves 37.3% on BrowseComp and 39.1% on BrowseComp-ZH. With context management, the scores further improve to 55.3% and 52.9%, respectively, significantly outperforming same-scale (4B) agents and even matching the performance of larger ones (approximately 30B). These results demonstrate the effectiveness of answer-backtracked step-level credit assignment for training long-horizon search agents.
科目:人工智能(cs.AI)
Subjects: Artificial Intelligence (cs.AI)