Kavli Affiliate: Feng Long| Summary: Federated low-rank adaptation (LoRA) requires identifying an update structure that is shared across heterogeneous clients. Prior work reports strong similarity among trained LoRA projection matrices across clients; however, such agreement may be largely induced by common initialization and collapses toward random overlap under independent initialization. More crucially, relying solely on […]
Continue.. What Should Federated LoRA Share? FedSAIL via Input-aware Subspace Alignment