如有疑问, 计划好: 致力于反应式强化学习的小语言模型审议

强化学习 (RL) 策略在不熟悉的环境中通常会退化，因为它们缺乏明确的深思熟虑。我们建议 Plan, Align, Commit, Think (PACT), 一种混合架构，它将快速, 反应式 RL 策略与慢速, 慎重小语言模型 (SLM) 规划器相结合。 PACT 异步调用 SLM 来生成和验证候选行动计划。一旦计划通过模拟验证为安全,、可行, 和完整,，它就会直接执行,，绕过 RL 策略，而无需重新训练或修改它。对三种难度不断增加的 FrozenLake 配置进行评估, PACT 优于所有基线，同时依赖 2B 参数 SLM 主干,，这表明在这些设置中，深思熟虑的计划和反应性执行比单独使用其中任何一个都更强大。

Reinforcement Learning (RL) policies often degrade in unfamiliar environments because they lack explicit deliberation. We propose Plan, Align, Commit, Think (PACT), a hybrid architecture that combines a fast, reactive RL policy with a slow, deliberative Small Language Model (SLM) planner. PACT invokes the SLM asynchronously to generate and validate candidate action plans. Once a plan is verified through simulation as safe, feasible, and complete, it is executed directly, bypassing the RL policy without retraining or modifying it. Evaluated on three FrozenLake configurations of increasing difficulty, PACT outperforms all baselines while relying on a 2B-parameter SLM backbone, suggesting that deliberative planning and reactive execution are more powerful in concert than either is alone in these settings.

科目: 人工智能 (cs.AI); 机器学习 (cs.LG)

Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)