Beyond Songs: Analyzing User Sentiment through Music Playlists and Multimodal Data
Open Access
- 20 February 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 21 (3) , 1-24
Abstract
The automatic recognition of user sentiments through their music listening behavior is an important research task in cognitive studies. Whereas prior studies were conducted to identify the sentiment conveyed (or evoked) by a song that a user listens to at a particular time, we argue that a more effective method would be to identify the user’s induced sentiment based on the comprehensive list of songs they have listened to (e.g., the sequence of music being played). However, recognizing the sentiment information induced by a playlist using machine learning techniques is much more challenging than identifying the sentiment induced by a single song, as it is difficult to obtain accurately labeled training samples for playlists. In this study, we developed the List–Song Relationship Factorization (LSRF) model with the objective of efficiently identifying sentiments induced by playlists. This model employs two side information constraints: the sentiment similarity between songs, based on multimodal information, and the co-occurrence of songs in playlists. These constraints enable the simultaneous co-clustering of songs and playlists. The experimental results demonstrate that the proposed model efficiently and consistently identifies sentiment information evoked by either playlists or individual songs.Keywords
Funding Information
- National Natural Science Foundation of China (91846105, 71671027, 72172092, 71772017, and 72293571)
- Natural Science Foundation of Sichuan Province (2024NSFSC0273)
- Innovative Research Team of Shanghai International Studies University (2020114044)
- Fundamental Research Funds for the Central Universities (2019114032)
This publication has 85 references indexed in Scilit:
- CNN and LSTM based ensemble learning for human emotion recognition using EEG recordingsMultimedia Tools and Applications, 2022
- A survey of music emotion recognitionFrontiers of Computer Science, 2022
- Music mood and human emotion recognition based on physiological signals: a systematic reviewMultimedia Systems, 2021
- Orthogonality, dependency, and music: An exploration of the relationships between music facetsJournal of the Association for Information Science and Technology, 2020
- A machine learning model for emotion recognition from physiological signalsBiomedical Signal Processing and Control, 2020
- Effects of recommendations on the playlist creation behavior of usersUser Modeling and User-Adapted Interaction, 2019
- Review of data features-based music emotion recognition methodsMultimedia Systems, 2017
- Task complexity and difficulty in music information retrievalJournal of the Association for Information Science and Technology, 2017
- A framework for evaluating multimodal music mood classificationJournal of the Association for Information Science and Technology, 2016
- Music emotion classification and context-based music recommendationMultimedia Tools and Applications, 2009