Kavli Affiliate: Jiansheng Chen | First 5 Authors: Kun Song, Zhiquan Tan, Bochao Zou, Jiansheng Chen, Huimin Ma | Summary: In this paper, we utilize information-theoretic metrics like matrix entropy and mutual information to analyze supervised learning. We explore the information content of data representations and classification head weights and their information interplay during supervised […]
Continue.. Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training