Kavli Affiliate: Feng Long| Summary: Low-rank adaptation (LoRA) has emerged as a powerful tool for parameter-efficient fine-tuning of large language models (LLMs). This paper studies LoRA under a federated learning setting, enabling collaborative fine-tuning across clients while preserving parameter efficiency. We focus on a highly heterogeneous regime in which clients share only partial structure and […]
Continue.. Federated LoRA Fine-Tuning for LLMs via Collaborative Alignment