Leveraging AI-powered interactive playbacks to decipher rules of communication in zebra finches
Authors:
- Logan S. JamesLJLogan S. James
- Benjamin HoffmanBHBenjamin Hoffman
- Jen-Yu LiuJLJen-Yu Liu
- Marius MironMMMarius Miron
- Milad AlizadehMAMilad Alizadeh
- Emmanuel FernandezEFEmmanuel Fernandez
- Matthieu GeistMGMatthieu Geist
- Jon T. SakataJSJon T. Sakata
- Emmanuel ChemlaECEmmanuel Chemla
- Olivier PietquinOPOlivier Pietquin
Preprint
Abstract
Vocal interactions are fundamental for social functioning across animals, including humans. The diverse rules underlying these exchanges remain largely unknown, and emerging AI technologies offer promising avenues for investigation. We used computational tools to collect and analyze > 1,000 hours of vocal interactions between female zebra finches and discovered that their interactions were characterized by correlated call production and structure, rapid acoustic modulation, and response selectivity. To test these interaction rules, we developed a generative audio large language model (ZF-AIM Acoustic Interaction Model) that engaged in real-time vocal exchanges with birds. When birds interacted with ZF-AIM, their vocal production and flexibility recapitulated key naturalistic features, which did not happen with non-interactive playbacks. Targeted ablations of ZF-AIM revealed that call timing and structure differentially contribute to natural vocal interactions. Using these AI-animal interactions, we demonstrate how AI can be leveraged to reveal fundamental rules underlying animal communication.Keywords
Funding Information
- Allen Family Philanthropies
This publication has 48 references indexed in Scilit:
- AI and the Doctor Dolittle challengeCurrent Biology, 2023
- Convergent behavioral strategies and neural computations during vocal turn-taking across diverse speciesCurrent Opinion in Neurobiology, 2022
- This time with feeling: learning expressive musical performanceNeural Computing and Applications, 2018
- The Forebrain Song System Mediates Predictive Call Timing in Female and Male Zebra FinchesCurrent Biology, 2016
- The vocal repertoire of the domesticated zebra finch: a data-driven approach to decipher the information-bearing acoustic features of communication signalsAnimal Cognition, 2015
- Impact of visual contact on vocal interaction dynamics of pair-bonded birdsAnimal Behaviour, 2015
- Coupled Oscillator Dynamics of Vocal Turn-Taking in MonkeysCurrent Biology, 2013
- A procedure for an automated measurement of song similarityAnimal Behaviour, 2000
- Evoked vocal response in male túngara frogs: pre-existing biases in male responses?Animal Behaviour, 1998
- Language-Independent Prosodic Features1983