Kavli Affiliate: Ke Wang
| First 5 Authors: Ke Wang, Guillermo Ortiz-Jimenez, Rodolphe Jenatton, Mark Collier, Efi Kokiopoulou
| Summary:
Label noise is a pervasive problem in deep learning that often compromises
the generalization performance of trained models. Recently, leveraging
privileged information (PI) — information available only during training but
not at test time — has emerged as an effective approach to mitigate this
issue. Yet, existing PI-based methods have failed to consistently outperform
their no-PI counterparts in terms of preventing overfitting to label noise. To
address this deficiency, we introduce Pi-DUAL, an architecture designed to
harness PI to distinguish clean from wrong labels. Pi-DUAL decomposes the
output logits into a prediction term, based on conventional input features, and
a noise-fitting term influenced solely by PI. A gating mechanism steered by PI
adaptively shifts focus between these terms, allowing the model to implicitly
separate the learning paths of clean and wrong labels. Empirically, Pi-DUAL
achieves significant performance improvements on key PI benchmarks (e.g., +6.8%
on ImageNet-PI), establishing a new state-of-the-art test set accuracy.
Additionally, Pi-DUAL is a potent method for identifying noisy samples
post-training, outperforming other strong methods at this task. Overall,
Pi-DUAL is a simple, scalable and practical approach for mitigating the effects
of label noise in a variety of real-world scenarios with PI.
| Search Query: ArXiv Query: search_query=au:”Ke Wang”&id_list=&start=0&max_results=3