Kavli Affiliate: Xiang Zhang
| First 5 Authors: Jiaqi Wei, Jiaqi Wei, , ,
| Summary:
Deliberative tree search is a cornerstone of modern Large Language Model
(LLM) research, driving the pivot from brute-force scaling toward algorithmic
efficiency. This single paradigm unifies two critical frontiers:
textbfTest-Time Scaling (TTS), which deploys on-demand computation to solve
hard problems, and textbfSelf-Improvement, which uses search-generated data
to durably enhance model parameters. However, this burgeoning field is
fragmented and lacks a common formalism, particularly concerning the ambiguous
role of the reward signal — is it a transient heuristic or a durable learning
target? This paper resolves this ambiguity by introducing a unified framework
that deconstructs search algorithms into three core components: the
emphSearch Mechanism, emphReward Formulation, and emphTransition
Function. We establish a formal distinction between transient textbfSearch
Guidance for TTS and durable textbfParametric Reward Modeling for
Self-Improvement. Building on this formalism, we introduce a component-centric
taxonomy, synthesize the state-of-the-art, and chart a research roadmap toward
more systematic progress in creating autonomous, self-improving agents.
| Search Query: ArXiv Query: search_query=au:”Xiang Zhang”&id_list=&start=0&max_results=3