AI 代理生态系统的快速发展正在改变复杂任务的委派和执行方式,，为给定任务识别合适的代理带来了新的挑战。与传统工具, 代理功能不同，, 代理功能通常是组合性的并且依赖于执行,，这使得它们很难仅通过文本描述进行评估。然而，, 现有的研究和基准测试通常假设明确的功能, 控制候选池, 或仅可执行任务查询, 未充分研究实际的代理搜索场景。我们引入了 AgentSearchBench,，这是一个针对野外代理搜索的大规模基准,，它由跨多个提供商的近 10,000 个真实代理构建而成。该基准将代理搜索形式化为可执行任务查询和高级任务描述,下的检索和重新排序问题，并使用基于执行的性能信号评估相关性。实验揭示了语义相似性和实际代理性能之间存在一致的差距,，暴露了基于描述的检索和重新排序方法的局限性。我们进一步表明，包括执行感知探测, 在内的轻量级行为信号, 可以显着提高排名质量,，强调将执行信号纳入代理发现的重要性。我们的代码可以在这个 https URL 上找到。
The rapid growth of AI agent ecosystems is transforming how complex tasks are delegated and executed, creating a new challenge of identifying suitable agents for a given task. Unlike traditional tools, agent capabilities are often compositional and execution-dependent, making them difficult to assess from textual descriptions alone. However, existing research and benchmarks typically assume well-specified functionalities, controlled candidate pools, or only executable task queries, leaving realistic agent search scenarios insufficiently studied. We introduce AgentSearchBench, a large-scale benchmark for agent search in the wild, built from nearly 10,000 real-world agents across multiple providers. The benchmark formalizes agent search as retrieval and reranking problems under both executable task queries and high-level task descriptions, and evaluates relevance using execution-grounded performance signals. Experiments reveal a consistent gap between semantic similarity and actual agent performance, exposing the limitations of description-based retrieval and reranking methods. We further show that lightweight behavioral signals, including execution-aware probing, can substantially improve ranking quality, highlighting the importance of incorporating execution signals into agent discovery. Our code is available at this https URL.
科目: 人工智能 (cs.AI); 信息检索 (cs.IR); 多代理系统 (cs.MA)
Subjects: Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Multiagent Systems (cs.MA)