三边市场中的调度为从世界反馈中强化学习提供了一个自然的环境: 决策是通过延迟的运营结果来评估的，例如交付速度, 快递利用率, 和商户拥堵。我们在 DoorDash 上展示了一个部署的强化学习系统，该系统使用延迟信号来调整大规模食品配送市场中的调度目标权重。从记录的市场数据中学习到的商店级策略不是取代组合分配优化器,，而是选择一个离散乘数，该乘数可以改变调度优化器的 在交付质量和批处理效率之间的权衡。该接口支持在嘈杂,延迟,和耦合反馈下进行离线策略学习，同时保留生产可行性约束和操作保障。我们使用集中式离线数据和分散式商店级执行,以及双 Q 学习目标和保守正则化器来训练共享价值函数，以减少分布外价值高估。在生产折返实验, 中，离线训练的策略增加了批量并降低了快递方的时间成本，而不会降低面向客户的交付质量。结果说明了如何利用来自实时经济和物流系统的世界反馈来安全地调整在线决策政策。
Dispatch in three-sided marketplaces provides a natural setting for reinforcement learning from world feedback: decisions are evaluated by delayed operational outcomes such as delivery speed, courier utilization, and merchant congestion. We present a deployed reinforcement learning system at DoorDash that adapts dispatch objective weights in a large-scale food-delivery marketplace using delayed signals. Rather than replacing the combinatorial assignment optimizer, a store-level policy learned from logged marketplace data selects a discrete multiplier that shifts the dispatch optimizer的 tradeoff between delivery quality and batching efficiency. This interface enables offline policy learning under noisy, delayed, and coupled feedback while preserving production feasibility constraints and operational safeguards. We train a shared value function using centralized offline data and decentralized store-level execution, with Double Q-learning targets and a conservative regularizer to reduce out-of-distribution value overestimation. In a production switchback experiment, the offline-trained policy increases batching and reduces courier-side time costs without degrading customer-facing delivery quality. Results illustrate how world feedback from a live economic and logistics system can be used to safely adapt decision policies online.
科目: 人工智能 (cs.AI); 机器学习 (cs.LG); 多代理系统 (cs.MA)
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multiagent Systems (cs.MA)