Kavli Affiliate: Yi Zhou
| First 5 Authors: Sheng Luo, Yi Zhou, Tao Zhou, ,
| Summary:
Incremental learning (IL) aims to overcome catastrophic forgetting of
previous tasks while learning new ones. Existing IL methods make strong
assumptions that the incoming task type will either only increases new classes
or domains (i.e. Class IL, Domain IL), or increase by a static scale in a
class- and domain-agnostic manner (i.e. Versatile IL (VIL)), which greatly
limit their applicability in the unpredictable and dynamic wild. In this work,
we investigate $textbf{Universal Incremental Learning (UIL)}$, where a model
neither knows which new classes or domains will increase along sequential
tasks, nor the scale of the increments within each task. This uncertainty
prevents the model from confidently learning knowledge from all task
distributions and symmetrically focusing on the diverse knowledge within each
task distribution. Consequently, UIL presents a more general and realistic IL
scenario, making the model face confusion arising from inter-task and
intra-task distribution randomness. To $textbf{Mi}$tigate both
$textbf{Co}$nfusion, we propose a simple yet effective framework for UIL,
named $textbf{MiCo}$. At the inter-task distribution level, we employ a
multi-objective learning scheme to enforce accurate and deterministic
predictions, and its effectiveness is further enhanced by a direction
recalibration module that reduces conflicting gradients. Moreover, at the
intra-task distribution level, we introduce a magnitude recalibration module to
alleviate asymmetrical optimization towards imbalanced class distribution.
Extensive experiments on three benchmarks demonstrate the effectiveness of our
method, outperforming existing state-of-the-art methods in both the UIL
scenario and the VIL scenario. Our code will be available at
$href{https://github.com/rolsheng/UIL}{here}$.
| Search Query: ArXiv Query: search_query=au:”Yi Zhou”&id_list=&start=0&max_results=3