Kavli Affiliate: Max Tegmark
| First 5 Authors: Ziming Liu, Mikail Khona, Ila R. Fiete, Max Tegmark,
| Summary:
Recurrent neural networks (RNNs) trained on compositional tasks can exhibit
functional modularity, in which neurons can be clustered by activity similarity
and participation in shared computational subtasks. Unlike brains, these RNNs
do not exhibit anatomical modularity, in which functional clustering is
correlated with strong recurrent coupling and spatial localization of
functional clusters. Contrasting with functional modularity, which can be
ephemerally dependent on the input, anatomically modular networks form a robust
substrate for solving the same subtasks in the future. To examine whether it is
possible to grow brain-like anatomical modularity, we apply a recent machine
learning method, brain-inspired modular training (BIMT), to a network being
trained to solve a set of compositional cognitive tasks. We find that
functional and anatomical clustering emerge together, such that functionally
similar neurons also become spatially localized and interconnected. Moreover,
compared to standard $L_1$ or no regularization settings, the model exhibits
superior performance by optimally balancing task performance and network
sparsity. In addition to achieving brain-like organization in RNNs, our
findings also suggest that BIMT holds promise for applications in neuromorphic
computing and enhancing the interpretability of neural network architectures.
| Search Query: ArXiv Query: search_query=au:”Max Tegmark”&id_list=&start=0&max_results=3