大型语言模型 (LLM) 推理已成为现代数据中心, 的主要工作负载，推动显着的 GPU 利用率和能耗。虽然现有系统通过批处理,、调度, 和并行, 来优化吞吐量和延迟，但它们在很大程度上将 GPU 功率视为静态约束而不是可控资源。在本文,中，我们为LLM服务, PALS,提供了一个功率感知运行时，它将GPU功率上限视为一流的控制旋钮，并与批量大小等软件参数联合优化它们。该系统将轻量级离线功率性能模型与反馈驱动控制器相结合，以选择满足吞吐量目标同时最大限度提高能源效率的配置。我们在现有的 LLM 服务框架, vLLM, 中实施 PALS，证明它不需要模型重新训练或 API 更改。在多 GPU 系统以及密集型和专家混合型 (MoE) 模型, PALS 中，能源效率提高了 26.3%, 在功率限制下将 QoS 违规降低了 4 到 7 倍, 并跟踪动态功率预算。这些结果凸显了将功率控制直接集成到 LLM 推理运行时间, 中的潜力，可实现能量比例和电网交互式 AI 系统。
Large language model (LLM) inference has become a dominant workload in modern data centers, driving significant GPU utilization and energy consumption. While prior systems optimize throughput and latency by batching, scheduling, and parallelism, they largely treat GPU power as a static constraint rather than a controllable resource. In this paper, we present a power-aware runtime for LLM serving, PALS, that treats GPU power caps as a first-class control knob and jointly optimizes them with software parameters such as batch size. The system combines lightweight offline power-performance models with a feedback-driven controller to select configurations that satisfy throughput targets while maximizing energy efficiency. We implement PALS within an existing LLM serving framework, vLLM, demonstrating that it requires no model retraining or API changes. Across multi-GPU systems and both dense and mixture-of-experts (MoE) models, PALS improves energy efficiency by up to 26.3%, reduces QoS violations by 4x to 7x under power constraints, and tracks dynamic power budgets. These results highlight the potential of integrating power control directly into LLM inference runtimes, enabling energy-proportional and grid-interactive AI systems.
科目:人工智能(cs.AI);分布式,并行,和集群计算(cs.DC)
Subjects: Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)