Kavli Affiliate: Yi Zhou
| First 5 Authors: Pengbo Hu, Ji Qi, Xingyu Li, Hong Li, Xinqi Wang
| Summary:
There emerges a promising trend of using large language models (LLMs) to
generate code-like plans for complex inference tasks such as visual reasoning.
This paradigm, known as LLM-based planning, provides flexibility in problem
solving and endows better interpretability. However, current research is mostly
limited to basic scenarios of simple questions that can be straightforward
answered in a few inference steps. Planning for the more challenging multi-hop
visual reasoning tasks remains under-explored. Specifically, under multi-hop
reasoning situations, the trade-off between accuracy and the complexity of
plan-searching becomes prominent. The prevailing algorithms either address the
efficiency issue by employing the fast one-stop generation or adopt a complex
iterative generation method to improve accuracy. Both fail to balance the need
for efficiency and performance. Drawing inspiration from the dual system of
cognition in the human brain, the fast and the slow think processes, we propose
a hierarchical plan-searching algorithm that integrates the one-stop reasoning
(fast) and the Tree-of-thought (slow). Our approach succeeds in performance
while significantly saving inference steps. Moreover, we repurpose the PTR and
the CLEVER datasets, developing a systematic framework for evaluating the
performance and efficiency of LLMs-based plan-search algorithms under reasoning
tasks at different levels of difficulty. Extensive experiments demonstrate the
superiority of our proposed algorithm in terms of performance and efficiency.
The dataset and code will be release soon.
| Search Query: ArXiv Query: search_query=au:”Yi Zhou”&id_list=&start=0&max_results=3