Kavli Affiliate: Max Tegmark | First 5 Authors: Ziming Liu, Max Tegmark, , , | Summary: Neural scaling laws (NSL) refer to the phenomenon where model performance improves with scale. Sharma & Kaplan analyzed NSL using approximation theory and predict that MSE losses decay as $N^{-alpha}$, $alpha=4/d$, where $N$ is the number of model parameters, […]

Kavli Affiliate: Max Tegmark | First 5 Authors: Ziming Liu, Max Tegmark, , , | Summary: Neural scaling laws (NSL) refer to the phenomenon where model performance improves with scale. Sharma & Kaplan analyzed NSL using approximation theory and predict that MSE losses decay as $N^{-alpha}$, $alpha=4/d$, where $N$ is the number of model parameters, […]

Kavli Affiliate: Max Tegmark | First 5 Authors: Wes Gurnee, Max Tegmark, , , | Summary: The capabilities of large language models (LLMs) have sparked debate over whether such systems just learn an enormous collection of superficial statistics or a coherent model of the data generating process — a world model. We find evidence for […]

Kavli Affiliate: Max Tegmark | First 5 Authors: Wes Gurnee, Max Tegmark, , , | Summary: The capabilities of large language models (LLMs) have sparked debate over whether such systems just learn an enormous collection of superficial statistics or a coherent model of the data generation process — a world model. We find preliminary evidence […]

Kavli Affiliate: Max Tegmark | First 5 Authors: Wes Gurnee, Max Tegmark, , , | Summary: The capabilities of large language models (LLMs) have sparked debate over whether such systems just learn an enormous collection of superficial statistics or a set of more coherent and grounded representations that reflect the real world. We find evidence […]