Kavli Affiliate: Max Tegmark | First 5 Authors: Anish Mudide, Joshua Engels, Eric J. Michaud, Max Tegmark, Christian Schroeder de Witt | Summary: Sparse autoencoders (SAEs) are a recent technique for decomposing neural network activations into human-interpretable features. However, in order for SAEs to identify all features represented in frontier models, it will be necessary […]
Kavli Affiliate: Max Tegmark | First 5 Authors: Yuxiao Li, Eric J. Michaud, David D. Baek, Joshua Engels, Xiaoqing Sun | Summary: Sparse autoencoders have recently produced dictionaries of high-dimensional vectors corresponding to the universe of concepts represented by large language models. We find that this concept universe has interesting structure at three levels: 1) […]
Kavli Affiliate: Ke Wang | First 5 Authors: Zimu Lu, Aojun Zhou, Ke Wang, Houxing Ren, Weikang Shi | Summary: Code has been shown to be effective in enhancing the mathematical reasoning abilities of large language models due to its precision and accuracy. Previous works involving continued mathematical pretraining often include code that utilizes math-related […]
Kavli Affiliate: Sara Seager | First 5 Authors: Amélie Gressier, Ryan J. MacDonald, Néstor Espinoza, Hannah R. Wakeford, Nikole K. Lewis | Summary: We present the first emission spectrum of the hot Jupiter WASP-17 b using one eclipse observation from the JWST Near Infrared Imager and Slitless Spectrograph (NIRISS) Single Object Slitless Spectroscopy (SOSS) mode. […]
Kavli Affiliate: Sara Seager | First 5 Authors: Daniel Valentine, Hannah R. Wakeford, Ryan C. Challener, Natasha E. Batalha, Nikole K. Lewis | Summary: We present the first spectroscopic characterisation of the dayside atmosphere of WASP-17b in the mid-infrared using a single JWST MIRI/LRS eclipse observation. From forward-model fits to the 5-12 $mu$m emission spectrum, […]
Kavli Affiliate: Wei Gao | First 5 Authors: Zhiyi Pan, Wei Gao, Shan Liu, Ge Li, | Summary: Despite alleviating the dependence on dense annotations inherent to fully supervised methods, weakly supervised point cloud semantic segmentation suffers from inadequate supervision signals. In response to this challenge, we introduce a novel perspective that imparts auxiliary constraints […]
Kavli Affiliate: Max Tegmark | First 5 Authors: David D. Baek, Yuxiao Li, Max Tegmark, , | Summary: Motivated by interpretability and reliability, we investigate how neural networks represent knowledge during graph learning, We find hints of universality, where equivalent representations are learned across a range of model sizes (from $10^2$ to $10^9$ parameters) and […]
Kavli Affiliate: Feng Wang | First 5 Authors: Feng Wang, Timing Yang, Yaodong Yu, Sucheng Ren, Guoyizhe Wei | Summary: In this work, we present a comprehensive analysis of causal image modeling and introduce the Adventurer series models where we treat images as sequences of patch tokens and employ uni-directional language models to learn visual […]
Kavli Affiliate: John E. Bowers | First 5 Authors: Shixin Xue, Mingxiao Li, Raymond Lopez-rios, Jingwei Ling, Zhengdong Gao | Summary: The invention of the laser unleashed the potential of optical metrology, leading to numerous advancements in modern science and technology. This reliance on lasers, however, also sets a bottleneck for precision optical metrology which […]
Kavli Affiliate: Lijing Shao | First 5 Authors: Alexander W. Raymond, Sheperd S. Doeleman, Keiichi Asada, Lindy Blackburn, Geoffrey C. Bower | Summary: The first very long baseline interferometry (VLBI) detections at 870$mu$m wavelength (345$,$GHz frequency) are reported, achieving the highest diffraction-limited angular resolution yet obtained from the surface of the Earth, and the highest-frequency […]