Kavli Affiliate: Feng Yuan
| First 5 Authors: Yichen He, Guanhua Huang, Peiyuan Feng, Yuan Lin, Yuchen Zhang
| Summary:
We introduce PaSa, an advanced Paper Search agent powered by large language
models. PaSa can autonomously make a series of decisions, including invoking
search tools, reading papers, and selecting relevant references, to ultimately
obtain comprehensive and accurate results for complex scholarly queries. We
optimize PaSa using reinforcement learning with a synthetic dataset,
AutoScholarQuery, which includes 35k fine-grained academic queries and
corresponding papers sourced from top-tier AI conference publications.
Additionally, we develop RealScholarQuery, a benchmark collecting real-world
academic queries to assess PaSa performance in more realistic scenarios.
Despite being trained on synthetic data, PaSa significantly outperforms
existing baselines on RealScholarQuery, including Google, Google Scholar,
Google with GPT-4 for paraphrased queries, chatGPT (search-enabled GPT-4o),
GPT-o1, and PaSa-GPT-4o (PaSa implemented by prompting GPT-4o). Notably,
PaSa-7B surpasses the best Google-based baseline, Google with GPT-4o, by 37.78%
in recall@20 and 39.90% in recall@50. It also exceeds PaSa-GPT-4o by 30.36% in
recall and 4.25% in precision. Model, datasets, and code are available at
https://github.com/bytedance/pasa.
| Search Query: ArXiv Query: search_query=au:”Feng Yuan”&id_list=&start=0&max_results=3