Deconfounded Cross-modal Matching for Content-based Micro-video Background Music Recommendation
Open Access
- 15 April 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 15 (3) , 1-25
Abstract
Object-oriented micro-video background music recommendation is a complicated task where the matching degree between videos and background music is a major issue. However, music selections in user-generated content (UGC) are prone to selection bias caused by historical preferences of uploaders. Since historical preferences are not fully reliable and may reflect obsolete behaviors, over-reliance on them should be avoided as knowledge and interests dynamically evolve. In this article, we propose a Deconfounded Cross-Modal matching model to mitigate such bias. Specifically, uploaders’ personal preferences of music genres are identified as confounders that spuriously correlate music embeddings and background music selections, causing the learned system to over-recommend music from majority groups. To resolve such confounders, backdoor adjustment is utilized to deconfound the spurious correlation between music embeddings and prediction scores. We further utilize Monte Carlo estimator with batch-level average as the approximations to avoid integrating the entire confounder space calculated by the adjustment. Furthermore, we design a teacher–student network to utilize the matching of music videos, which is professionally generated content (PGC) with specialized matching, to better recommend content-matching background music. The PGC data are modeled by a teacher network to guide the matching of uploader-selected UGC data of student network by Kullback–Leibler–based knowledge transfer. Extensive experiments on the TT-150k-genre dataset demonstrate the effectiveness of the proposed method. The code is publicly available on https://github.com/jing-1/DecCMKeywords
Funding Information
- National Natural Science Foundation of China (62036005)
This publication has 22 references indexed in Scilit:
- Bias and Debias in Recommender System: A Survey and Future DirectionsACM Transactions on Information Systems, 2023
- Mitigating Popularity Bias for Users and Items with Fairness-centric Adaptive RecommendationACM Transactions on Information Systems, 2023
- Addressing Confounding Feature Issue for Causal RecommendationACM Transactions on Information Systems, 2023
- Video-to-Music Recommendation Using Temporal Alignment of SegmentsIEEE Transactions on Multimedia, 2022
- Cross-Modal Variational Auto-Encoder for Content-Based Micro-Video Background Music RecommendationIEEE Transactions on Multimedia, 2021
- A Survey on Causal InferenceACM Transactions on Knowledge Discovery From Data, 2021
- Background music recommendation based on latent factors and moodsKnowledge-Based Systems, 2018
- Beyond RelevanceACM Transactions on Intelligent Systems and Technology, 2016
- Infotopia How Many Minds Produce Knowledge2006
- The Monte Carlo MethodJournal of the American Statistical Association, 1949