Kavli Affiliate: Zhuo Li
| First 5 Authors: Zhuo Li, Guodong Du, Weiyang Guo, Yigeng Zhou, Xiucheng Li
| Summary:
Aligning large language models (LLMs) to simultaneously satisfy multiple
objectives remains a significant challenge, especially given the diverse and
often conflicting nature of human preferences. Existing alignment methods
struggle to balance trade-offs effectively, often requiring costly retraining
or yielding suboptimal results across the Pareto frontier of preferences. In
this paper, we introduce textit{HoE}(Hierarchical Mixture-of-Experts), a
textit{lightweight}, textit{parameter-efficient}, and textit{plug-and-play}
approach that eliminates the need for model training, while enabling LLMs to
adapt across the entire Pareto frontier and accommodate diverse user
preferences. In particular, textit{HoE} consists of three hierarchical
components: LoRA Experts, Router Experts and Preference Routing, reaching
optimal Pareto frontiers and achieving a trade-off between parameter size,
training cost, and performance. We evaluate textit{HoE} across various tasks
on 14 objectives and 200 different preferences among 6 benchmarks,
demonstrating superior performance over 15 recent baselines. Code is available
in the supplementary materials.
| Search Query: ArXiv Query: search_query=au:”Zhuo Li”&id_list=&start=0&max_results=3