Kavli Affiliate: Zheng Zhu
| First 5 Authors: Zekai Li, Xinhao Zhong, Samir Khaki, Zhiyuan Liang, Yuhao Zhou
| Summary:
In recent years, dataset distillation has provided a reliable solution for
data compression, where models trained on the resulting smaller synthetic
datasets achieve performance comparable to those trained on the original
datasets. To further improve the performance of synthetic datasets, various
training pipelines and optimization objectives have been proposed, greatly
advancing the field of dataset distillation. Recent decoupled dataset
distillation methods introduce soft labels and stronger data augmentation
during the post-evaluation phase and scale dataset distillation up to larger
datasets (e.g., ImageNet-1K). However, this raises a question: Is accuracy
still a reliable metric to fairly evaluate dataset distillation methods? Our
empirical findings suggest that the performance improvements of these methods
often stem from additional techniques rather than the inherent quality of the
images themselves, with even randomly sampled images achieving superior
results. Such misaligned evaluation settings severely hinder the development of
DD. Therefore, we propose DD-Ranking, a unified evaluation framework, along
with new general evaluation metrics to uncover the true performance
improvements achieved by different methods. By refocusing on the actual
information enhancement of distilled datasets, DD-Ranking provides a more
comprehensive and fair evaluation standard for future research advancements.
| Search Query: ArXiv Query: search_query=au:”Zheng Zhu”&id_list=&start=0&max_results=3