Matthieu Geist
Research Subjects
Matthieu Geist has a total of 31 co-authors. Here are the top co-authors by number of shared publications:
- Benjamin Hoffman
Earth Species Project
3 shared publications - Emmanuel Chemla
Laboratoire de Sciences Cognitives et Psycholinguistique, Département d’Études Cognitives, ENS, PSL University, EHESS, CNRS, Paris, France
3 shared publications - Logan S. James
Department of Biology, McGill University, Montreal, Quebec, Canada
3 shared publications - Marius Miron
Music Technology Group, Universitat Pompeu Fabra, Barcelona, Spain
3 shared publications - Milad Alizadeh
Earth Species Project
3 shared publications - Jen-Yu Liu
Earth Species Project, Berkeley, CA
2 shared publications
Latest Publications
- Open Access
BirdCODE: Detecting bird communication at scale
Deep learning-based animal sound identification is regularly applied to large audio datasets for ecological monitoring and citizen science, but existing methods lack the fine temporal resolution required to extract insights into animal communication from these same datasets. Here we introduce Bird Communication Detector (BirdCODE), a deep learning model that detects and classifies the vocalizations of over 9000 bird species with precise temporal boundaries, a several hundredfold increase the number of species over previous bioacoustic sound event detection models. In extensive benchmarking, BirdCODE achieves state-of-the-art performance in detection and classification of bird sounds. Applying BirdCODE to 1.3M citizen-science recordings, we present four case studies of how BirdCODE-computed sound event boundaries can be used to carry out phylogenetic analyses, to describe geographic and temporal variation in acoustic communication, and to characterize cross-species interactions. Together, these demonstrate how BirdCODE can enable large-scale, data-driven studies of bird communication. Model code, weights, and predictions are publicly available.
Open AccessBirdCODE: Detecting bird communication at scale
Deep learning-based animal sound identification is regularly applied to large audio datasets for ecological monitoring and citizen science, but existing methods lack the fine temporal resolution required to extract insights into animal communication from these same datasets. Here we introduce Bird Communication Detector (BirdCODE), a deep learning model that detects and classifies the vocalizations of over 9000 bird species with precise temporal boundaries, a several hundredfold increase the number of species over previous bioacoustic sound event detection models. In extensive benchmarking, BirdCODE achieves state-of-the-art performance in detection and classification of bird sounds. Applying BirdCODE to 1.3M citizen-science recordings, we present four case studies of how BirdCODE-computed sound event boundaries can be used to carry out phylogenetic analyses, to describe geographic and temporal variation in acoustic communication, and to characterize cross-species interactions. Together, these demonstrate how BirdCODE can enable large-scale, data-driven studies of bird communication. Model code, weights, and predictions are publicly available.
- Open Access
Repertoire-behavior mapping reveals signal functions in cooperatively breeding crows
Communication structures society, and is likewise shaped by relationships and shared tasks; yet, for most socially complex species, we know little of their full vocal repertoire and its functions. We investigated how communication structures the who, what, and when of social interactions in cooperative carrion crows – group-living birds who rely on coordinated behaviors, as in chick care. Leveraging machine learning to integrate large-scale data from crow-borne audio-loggers and nest cameras, we charted the vocal repertoire across 24 cooperative groups and mapped all discovered call types to behaviors and social context. We found that crows used a rich repertoire across three domains of joint behavior – flocking, chick care, and territorial display. Relatively quiet call types were abundant and included close-range calls that may coordinate chick care by announcing nest visits. Our study demonstrates how combining continuous-capture data and machine learning can reveal a holistic understanding of how vocalizations function across contexts.
Open AccessRepertoire-behavior mapping reveals signal functions in cooperatively breeding crows
Communication structures society, and is likewise shaped by relationships and shared tasks; yet, for most socially complex species, we know little of their full vocal repertoire and its functions. We investigated how communication structures the who, what, and when of social interactions in cooperative carrion crows – group-living birds who rely on coordinated behaviors, as in chick care. Leveraging machine learning to integrate large-scale data from crow-borne audio-loggers and nest cameras, we charted the vocal repertoire across 24 cooperative groups and mapped all discovered call types to behaviors and social context. We found that crows used a rich repertoire across three domains of joint behavior – flocking, chick care, and territorial display. Relatively quiet call types were abundant and included close-range calls that may coordinate chick care by announcing nest visits. Our study demonstrates how combining continuous-capture data and machine learning can reveal a holistic understanding of how vocalizations function across contexts.
- Open Access
Leveraging AI-powered interactive playbacks to decipher rules of communication in zebra finches
Vocal interactions are fundamental for social functioning across animals, including humans. The diverse rules underlying these exchanges remain largely unknown, and emerging AI technologies offer promising avenues for investigation. We used computational tools to collect and analyze > 1,000 hours of vocal interactions between female zebra finches and discovered that their interactions were characterized by correlated call production and structure, rapid acoustic modulation, and response selectivity. To test these interaction rules, we developed a generative audio large language model (ZF-AIM Acoustic Interaction Model) that engaged in real-time vocal exchanges with birds. When birds interacted with ZF-AIM, their vocal production and flexibility recapitulated key naturalistic features, which did not happen with non-interactive playbacks. Targeted ablations of ZF-AIM revealed that call timing and structure differentially contribute to natural vocal interactions. Using these AI-animal interactions, we demonstrate how AI can be leveraged to reveal fundamental rules underlying animal communication.
Open AccessLeveraging AI-powered interactive playbacks to decipher rules of communication in zebra finches
Vocal interactions are fundamental for social functioning across animals, including humans. The diverse rules underlying these exchanges remain largely unknown, and emerging AI technologies offer promising avenues for investigation. We used computational tools to collect and analyze > 1,000 hours of vocal interactions between female zebra finches and discovered that their interactions were characterized by correlated call production and structure, rapid acoustic modulation, and response selectivity. To test these interaction rules, we developed a generative audio large language model (ZF-AIM Acoustic Interaction Model) that engaged in real-time vocal exchanges with birds. When birds interacted with ZF-AIM, their vocal production and flexibility recapitulated key naturalistic features, which did not happen with non-interactive playbacks. Targeted ablations of ZF-AIM revealed that call timing and structure differentially contribute to natural vocal interactions. Using these AI-animal interactions, we demonstrate how AI can be leveraged to reveal fundamental rules underlying animal communication.