Kavli Affiliate: Feng Wang
| First 5 Authors: Jiaxing Li, Chi Xu, Lianchen Jia, Feng Wang, Cong Zhang
| Summary:
Large language models (LLMs) have demonstrated impressive capabilities in
language tasks, but they require high computing power and rely on static
knowledge. To overcome these limitations, Retrieval-Augmented Generation (RAG)
incorporates up-to-date external information into LLMs without extensive
fine-tuning. Meanwhile, small language models (SLMs) deployed on edge devices
offer efficiency and low latency but often struggle with complex reasoning
tasks. Unfortunately, current RAG approaches are predominantly based on
centralized databases and have not been adapted to address the distinct
constraints associated with deploying SLMs in edge environments. To bridge this
gap, we propose Edge-Assisted and Collaborative RAG (EACO-RAG), a lightweight
framework that leverages distributed edge nodes for adaptive knowledge updates
and retrieval. EACO-RAG also employs a hierarchical collaborative gating
mechanism to dynamically select among local, edge-assisted, and cloud-based
strategies, with a carefully designed algorithm based on Safe Online Bayesian
Optimization to maximize the potential performance enhancements. Experimental
results demonstrate that EACO-RAG matches the accuracy of cloud-based knowledge
graph RAG systems while reducing total costs by up to 84.6% under relaxed delay
constraints and by 65.3% under stricter delay requirements. This work
represents our initial effort toward achieving a distributed and scalable
tiered LLM deployments, with EACO-RAG serving as a promising first step in
unlocking the full potential of hybrid edge-cloud intelligence.
| Search Query: ArXiv Query: search_query=au:”Feng Wang”&id_list=&start=0&max_results=3