LLM 智能体能否在没有梯度更新的情况下通过自我生成的记忆来改进决策? 我们提出 FORGE (Failure-Optimized Reflective Graduation and Evolution), 一个分阶段的, 基于群体的协议，为分层 ReAct 智能体发展提示注入的自然语言记忆。 FORGE 包装了反射式内部循环,，其中专用反射代理 ( 使用相同的基础 LLM, 没有从更强的模型中蒸馏) 将失败的轨迹转换为可重用的知识工件: 文本启发式 (Rules), 少量演示 (Examples), 或两者 (Mixed), 具有传播的外部循环将性能最好的实例的内存分配给阶段之间的群体，并通过分级标准冻结聚合实例。我们在 CybORG CAGE-2, 上针对 B 线攻击者, 在 30 步范围内评估随机网络防御 POMDP，其中所有四个测试的 LLM 系列 (Gemini-2.5-Flash-Lite, Grok-4-Fast, Llama-4-Maverick, Qwen3-235B) 都表现出强烈的负, 重尾零样本奖励。与零样本基线和反射基线相比(隔离单流学习), FORGE 在所有 12 个模型表示条件下将平均评估回报比零样本提高了 1.7-7.7$\ 倍$，比反射提高了 29-72%, 降低了重大故障率(低于$-100$)低至 $\sim$1%。我们发现 (1) 群体广播是关键机制, 采用无分级消融，确认广播带来性能增益，而分级主要节省计算; (2) 示例为四个模型中的三个实现了最强回报, 规则提供了最佳的成本可靠性概况，减少了 $\sim$40% 的代币; 和 (3) 较弱的基线模型受益不成比例, 表明 FORGE 可以缩小能力差距而不是放大强大的模型。所有证据仅限于 CAGE-2 B 线; 跨家族发现是方向性证据。

Can LLM agents improve decision-making through self-generated memory without gradient updates? We propose FORGE (Failure-Optimized Reflective Graduation and Evolution), a staged, population-based protocol that evolves prompt-injected natural-language memory for hierarchical ReAct agents. FORGE wraps a Reflexion-style inner loop, where a dedicated reflection agent (using the same underlying LLM, no distillation from a stronger model) converts failed trajectories into reusable knowledge artifacts: textual heuristics (Rules), few-shot demonstrations (Examples), or both (Mixed), with an outer loop that propagates the best-performing instance的 memory to the population between stages and freezes converged instances via a graduation criterion. We evaluate on CybORG CAGE-2, a stochastic network-defense POMDP at a 30-step horizon against the B-line attacker, where all four tested LLM families (Gemini-2.5-Flash-Lite, Grok-4-Fast, Llama-4-Maverick, Qwen3-235B) exhibit strongly negative, heavy-tailed zero-shot rewards. Compared against both a zero-shot baseline and a Reflexion baseline (isolated single-stream learning), FORGE improves average evaluation return by 1.7-7.7$\times$ over zero-shot and by 29-72% over Reflexion in all 12 model-representation conditions, reducing major-failure rates (below $-100$) to as low as $\sim$1%. We find that (1) population broadcast is critical mechanism, with a no-graduation ablation confirming that broadcast carries the performance gains while graduation primarily saves compute; (2) Examples achieves the strongest returns for three of four models, Rules offers the best cost-reliability profile with $\sim$40% fewer tokens; and (3) weaker baseline models benefit disproportionately, suggesting FORGE may mitigate capability gaps rather than amplify strong models. All evidence is confined to CAGE-2 B-line; cross-family findings are directional evidence.

科目: 人工智能 (cs.AI); 计算和语言 (cs.CL); 机器学习 (cs.LG); 多代理系统 (cs.MA); 系统和控制 (eess.SY)

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multiagent Systems (cs.MA); Systems and Control (eess.SY)