Kavli Affiliate: Max Tegmark | First 5 Authors: Ziming Liu, Ouail Kitouni, Niklas Nolte, Eric J. Michaud, Max Tegmark | Summary: We aim to understand grokking, a phenomenon where models generalize long after overfitting their training set. We present both a microscopic analysis anchored by an effective theory and a macroscopic analysis of phase diagrams […]
Continue.. Towards Understanding Grokking: An Effective Theory of Representation Learning