OLOlivier Lartillot

Olivier Lartillot

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Olivier Lartillot has a total of 33 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    A Multimodal Dataset of Greek Folk Music

    by Anna-Maria Christodoulou, Olivier Lartillot


  • Open Access

    MusiQAl: A dataset for Music Question-Answering through Audio-Video Fusion

    by Anna-Maria Christodoulou, Kyrre Glette, Olivier Lartillot, Alexander Refsum Jensenius

    Music Question--Answering (MQA) is a machine learning task where a computational system analyzes and answers questions about music-related data. Traditional methods prioritize audio, overlooking visual and embodied aspects crucial to music performance understanding. We introduce MusiQAl, a multimodal dataset of 310 music performance videos and 11,793 human-annotated question--answer pairs, spanning diverse musical traditions and styles. Grounded in musicology and music psychology, MusiQAl emphasizes multimodal reasoning, causal inference, and cross-cultural understanding of performer-music interactions. We benchmark AVST and LAVISH on MusiQAl, revealing strengths and limitations, underscoring the importance of integrating multimodal learning and domain expertise to advance MQA and Music Information Retrieval (MIR).

  • Open Access

    Video Visualization of a String Quartet Performance of a Bach Fugue: Design and Subjective Evaluation

    by Olivier Lartillot, Dana Swarbrick, Finn Upham, Carlos Eduardo Cancino-Chacón

    Visualizing music—through music notation, analytical representations, or music videos—might potentially boost the appreciation of music in all its richness. The purpose of this study was to design and test a visualization strategy aimed at explicating to a large audience with diverse backgrounds—especially novices—the multifaceted beauty of the final Contrapunctus in J.S. Bach's The Art of Fugue , performed by the Danish String Quartet. At the surface level of the musical structure, the rich fluctuation of pitch shaped by each musician was depicted in the form of undulating pitch curves. At a deeper structural level, the repetition of pitch curves, distinctive of fugues, was highlighted through vertical alignment—inspired by a technique called paradigmatic analysis, originating from anthropology and music semiology. The visualization was initially prototyped in the form of a real-time technology as part of the MusicLab Copenhagen research concert. The concert audience focused on the performance itself, and did not pay much attention to, nor appreciate, the visualization. To evaluate more thoroughly the potential of the visualization, participants with varied musical expertise and taste were invited to listen to a recorded performance of the piece and watch the visualization on their own computer. A large majority reported that they felt they understood the visualization, around half of them felt that it enhanced their musical understanding, and a small group felt that it helped them to better appreciate the music.

  • Open Access

    Perspectives on digital music edition, heritage preservation, annotation and analysis: a COST Action meeting in Barcelona

    by Emilio Ros-Fábregas, Kevin Page, Mark Saccomano, Olivier Lartillot, Anastasiia Mazurenko

    This article brings together the contribution of the participants in a Short-Term Scientific Mission (STSM) of the COST Action EarlyMuse (CA21161) which took place at the IMF-CSIC in Barcelona (June 2024). The discussions involved different aspects of projects with digital music edition, annotation, analysis, and the problems related to preservation of music heritage, with particular attention to the situation in Ukraine. The diverse geographical and professional background and expertise of the authors provides, in their respective sections of the article, a rich panorama on technological projects, issues, models, interoperability, and digital tools. A final Appendix contains a useful list, with links, of the projects, websites, and tools mentioned during the STSM meeting.


  • Open Access

    Multimodal Music Datasets? Challenges and Future Goals in Music Processing

    by Anna-Maria Christodoulou, Olivier Lartillot, Alexander Refsum Jensenius

    The term ``multimodal music dataset'' is often used to describe music-related data that represent music as a multimedia art form and multimodal experience. However, the term ``multimodality'' may mean different things in related disciplines, such as musicology, music psychology, and music technology. This paper proposes a definition of multimodality that works across different music disciplines. Many challenges are related to constructing, evaluating, and using multimodal music datasets. We provide a task-based categorization of these datasets and explore theoretical methodologies aimed at enhancing their future construction. Diverse data preprocessing methods are illuminated, highlighting their contributions to transparent music analysis. Additionally, evaluation metrics, methods, and benchmarks tailored for multimodal music processing tasks are scrutinized, empowering researchers to make informed decisions and facilitating cross-study comparisons.


  • Open Access

    A Dataset of Norwegian Hardanger Fiddle Recordings with Precise Annotation of Note and Beat Onsets

    by Olivier Lartillot, Mats Sigvard Johansson, Anders Elowsson, Lars Løberg Monstad, Mattias Cyvin

    The Hardanger fiddle is a variety of the violin used in the folk music of the western and central part of southern Norway. This paper presents a dataset of several hours of recordings of Hardanger fiddle music, with note annotations of onsets, offsets and pitches, provided by the performers themselves. A subset has also been annotated with beat onset positions by the performer as well as three expert musicians. The complexity of the music genre—polyphonic, highly ornamented and with a very irregular pulsation, among other aspects—motivated the design of a new annotation software adapted to these particular needs. Beat annotation in MIR is typically recorded as positions in seconds, without explicit connection with actual musical events. In the context of music where the rhythm is carried by the melodic instrument alone, a more reliable definition of beat onsets consists in associating them with the onsets of the notes that represent the start of each beat. This latter definition of beat onsets reflects that beats are generated from within the flow of played melodic-rhythmic events, which implies that the spacing of beats may be shifting and irregular. This motivated the design of a new method for beat annotation in Hardanger fiddle music based on a selection of notes in the note annotation. Comparisons between annotators through alignment—integrated in the interface—enable them to eventually correct their annotations or observe alternative valid interpretations of any given excerpt. After dedicating a part of the note annotation dataset to the training of a machine learning model, for the task of assessing both note pitch and onset time, an F1 score of 87% can be reached. The beat annotation dataset demonstrates the necessity of developing new beat trackers adapted to Hardanger fiddle music. The dataset as well as the annotation software is made publicly available.


  • Open Access

    Mental Effort and Expressive Interaction in Expert and Student String Quartet Performance

    by Laura Bishop, Simon Høffding, Olivier Lartillot, Bruno Laeng

    Resilience to changes in performance environments is a hallmark of expertise in music performance. Research has shown that skilled non-expert ensemble musicians maintain synchronization when visual interaction between them is disrupted, but that the quality of their expressive body motion changes. Our study extended these findings by testing how an expert string quartet responds to playing conditions that disrupt visual contact. We tested for potential effects on the musicians’ expressive head motion, sound quality, and mental effort. The Danish String Quartet (DSQ), a world-class classical ensemble, performed an excerpt from a Haydn piece five times without an audience, then once for an audience of about 20 people. During the performances without audience, their seating configuration was manipulated to disrupt their audiovisual interaction. Audio, head motion, eye-tracking, and pupillometry data were collected. The DSQ's data were compared to data from a student quartet who completed the same experiment. Our results show that the DSQ maintained the quality of their musical sound and interactive body motion across disruptive and non-disruptive conditions, but mental effort (indexed by pupil size) was greater in non-disruptive conditions. In contrast, the student quartet moved less overall than the DSQ, moved less when they could not see each other, and did not show differences in pupil size across conditions. The quartets spent a similar percentage of performance time watching their co-performers. These findings suggest that the quality of audio and visual components of the DSQ's performance do not require visual interaction to maintain; however, these musicians do interact visually when given the opportunity. This visual interaction stimulates greater mental effort, perhaps reflecting increased social engagement.


  • Computational localization of attack regions through a direct observation of the audio waveform

    by Olivier Lartillot, Kristian Nymoen, Guilherme Schmidt Câmara, Anne Danielsen

    This article addresses the computational estimation of attack regions in audio recordings. Previous attempts to do so were based on the reduction of the audio waveform into an envelope curve, which decreases its temporal resolution. The proposed approach detects the attack region directly from the audio waveform. The attack region is modeled as a line starting from a low-amplitude point and intersecting one of the local maxima according to two principles: (1) maximizing the slope, while favoring, at the same time, a higher peak if the slope remains only slightly lower and (2) dismissing initial attack regions of relatively low amplitude. The attack start position is fine-tuned by intersecting the attack slope with the audio waveform. The proposed method precisely pinpoints the attack region in cases where it is unambiguously observable from the waveform itself. In such cases, previous methods selected a broader attack region due to the loss of temporal resolution. When attack regions are less evident, the proposed method's estimation remains within the range of results provided by other methods. Applied to the prediction of judgments of P-center localization [Danielsen, Nymoen, Anderson, Câmara, Langerød, Thompson, and London, J. Exp. Psychol. Hum. Percept. Perform. 45, 402–418 (2019)], the proposed method shows a significant increase in precision, at the expense of recall.


  • Open Access

    Tempo and Metrical Analysis by Tracking Multiple Metrical Levels Using Autocorrelation

    by Olivier Lartillot, Didier Grandjean

    We present a method for tempo estimation from audio recordings based on signal processing and peak tracking, and not depending on training on ground-truth data. First, an accentuation curve, emphasizing the temporal location and accentuation of notes, is based on a detection of bursts of energy localized in time and frequency. This enables the detection of notes in dense polyphonic texture, while ignoring spectral fluctuation produced by vibrato and tremolo. Periodicities in the accentuation curve are detected using an improved version of autocorrelation function. Hierarchical metrical structures, composed of a large set of periodicities in pairwise harmonic relationships, are tracked over time. In this way, the metrical structure can be tracked even if the rhythmical emphasis switches from one metrical level to another. This approach, compared to all the other participants to the Music Information Retrieval Evaluation eXchange (MIREX) Audio Tempo Extraction competition from 2006 to 2018, is the third best one among those that can track tempo variations. While the two best methods are based on machine learning, our method suggests a way to track tempo founded on signal processing and heuristics-based peak tracking. Moreover, the approach offers for the first time a detailed representation of the dynamic evolution of the metrical structure. The method is integrated into MIRtoolbox, a Matlab toolbox freely available.