随着生成长度的不断增长，稀疏注意力对于服务大型语言模型 (LLMs) 变得越来越重要。然而，, 大规模部署和评估新的稀疏注意力算法仍然是高度工程密集型的,，减缓了人类研究人员和人工智能代理探索稀疏注意力设计的速度。为了应对这一挑战,，我们提出了 Vortex, 一个系统，该系统将 Python 嵌入式前端语言与以页面为中心的张量抽象相结合，用于表达广泛的稀疏注意力算法, 与紧密集成到现代 LLM 服务堆栈中的高效后端。 Vortex 支持快速原型, 部署, 和稀疏注意力算法, 的评估，有效地将其理论效率增益转化为现实世界的吞吐量改进。因此，, Vortex 极大地加速了稀疏注意力算法的设计和迭代。首先, AI 代理使用 Vortex 自动生成和完善不同的算法,，最好的吞吐量比完全注意力提高高达 $3.46\ 倍$，同时保持准确性。 Second, Vortex 将稀疏注意力扩展到新兴架构和非常大的模型上，否则很难进行实验，, 在基于 MLA 的 GLM-4.7-Flash 上吞吐量提高了 $4.7\times$，在 NVIDIA B200 GPU 上的 229B 参数 MiniMax-M2.7 上吞吐量提高了 $1.37\time$。
Sparse attention is becoming increasingly important for serving large language models (LLMs) as generation lengths continue to grow. However, deploying and evaluating new sparse attention algorithms at scale remains highly engineering-intensive, slowing both human researchers and AI agents in exploring the sparse attention design. To address this challenge, we present Vortex, a system that combines a Python-embedded frontend language atop a page-centric tensor abstraction for expressing a broad range of sparse attention algorithms, with an efficient backend tightly integrated into modern LLM serving stacks. Vortex enables rapid prototyping, deployment, and evaluation of sparse attention algorithms, effectively translating their theoretical efficiency gains into real-world throughput improvements. As a result, Vortex substantially accelerates the design and iteration of sparse attention algorithms. First, AI agents use Vortex to automatically generate and refine diverse algorithms, the best reaching up to $3.46\times$ higher throughput than full attention while preserving accuracy. Second, Vortex extends sparse attention to emerging architectures and very large models that are otherwise hard to experiment with, reaching up to $4.7\times$ higher throughput on the MLA-based GLM-4.7-Flash and $1.37\times$ on the 229B-parameter MiniMax-M2.7 on NVIDIA B200 GPUs.
科目:人工智能(cs.AI)
Subjects: Artificial Intelligence (cs.AI)