Kavli Affiliate: Sarah Woolley
| Authors: Logan S James, Benjamin Hoffman, Jen-Yu Liu, Marius Miron, Milad Alizadeh, Emmanuel Fernandez, Matthieu Geist, Diane Kim, Aza Raskin, Jon T Sakata, Emmanuel Chemla, Olivier Pietquin and Sarah C Woolley
| Summary:
Vocal interactions are fundamental for social functioning across animals, including humans. The diverse rules underlying these exchanges remain largely unknown, and emerging AI technologies offer promising avenues for investigation. We used computational tools to collect and analyze >1,000 hours of vocal interactions between female zebra finches and discovered that their interactions were characterized by correlated call production and structure, rapid acoustic modulation, and response selectivity. To test these interaction rules, we developed a generative audio large language model (ZF-AIM Acoustic Interaction Model) that engaged in real-time vocal exchanges with birds. When birds interacted with ZF-AIM, their vocal production and flexibility recapitulated key naturalistic features, which did not happen with non-interactive playbacks. Targeted ablations of ZF-AIM revealed that call timing and structure differentially contribute to natural vocal interactions. Using these AI-animal interactions, we demonstrate how AI can be leveraged to reveal fundamental rules underlying animal communication.