Kavli Affiliate: Max Tegmark | First 5 Authors: Eric J. Michaud, Ziming Liu, Uzay Girit, Max Tegmark, | Summary: We propose the Quantization Model of neural scaling laws, explaining both the observed power law dropoff of loss with model and data size, and also the sudden emergence of new capabilities with scale. We derive this […]
Continue.. The Quantization Model of Neural Scaling