Kavli Affiliate: Feng Wang
| First 5 Authors: Zheqi Lv, Feng Wang, Kun Kuang, Yongwei Wang, Zhengyu Chen
| Summary:
Deploying machine learning models on mobile devices has gained increasing
attention. To tackle the model generalization problem with the limitations of
hardware resources on the device, the device model needs to be lightweight by
techniques such as model compression from the cloud model. However, the major
obstacle to improve the device model generalization is the distribution shift
between the data of cloud and device models, since the data distribution on
device model often changes over time (e.g., users might have different
preferences in recommendation system). Although real-time fine-tuning and
distillation method take this situation into account, these methods require
on-device training, which are practically infeasible due to the low
computational power and a lack of real-time labeled samples on the device.
In this paper, we propose a novel task-agnostic framework, named MetaNetwork,
for generating adaptive device model parameters from cloud without on-device
training. Specifically, our MetaNetwork is deployed on cloud and consists of
MetaGenerator and MetaStabilizer modules. The MetaGenerator is designed to
learn a mapping function from samples to model parameters, and it can generate
and deliver the adaptive parameters to the device based on samples uploaded
from the device to the cloud. The MetaStabilizer aims to reduce the oscillation
of the MetaGenerator, accelerate the convergence and improve the model
performance during both training and inference. We evaluate our method on two
tasks with three datasets. Extensive experiments show that MetaNetwork can
achieve competitive performances in different modalities.
| Search Query: ArXiv Query: search_query=au:”Feng Wang”&id_list=&start=0&max_results=10