Kavli Affiliate: Ke Wang | First 5 Authors: Ke Wang, Nikolaos Dimitriadis, Alessandro Favero, Guillermo Ortiz-Jimenez, Francois Fleuret | Summary: Large pre-trained models exhibit impressive zero-shot performance across diverse tasks, but fine-tuning often leads to catastrophic forgetting, where improvements on a target domain degrade generalization on other tasks. To address this challenge, we introduce LiNeS, […]
Continue.. LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging