Multimodal Manga Tag Estimation Using Image, Text, and Emotion Features
Preprint
- 27 October 2026
- preprint
- Published by MDPI AG in Preprints.org
Abstract
Manga conveys its content through multiple modalities, including page images, dialogue, and emotional progression throughout a story. This study investigates how image, text, and emotion information contribute to volume-level manga tag estimation and whether combining these modalities improves estimation performance. We estimate three tag types–publication decade, target readership, and genre–for 109 volumes in the Manga109 dataset. We extract image features from two-page spread images using CLIP, derive text features from the lines contained in each spread, and obtain emotion features from valence–arousal emotion arcs generated by a large multimodal model. We examine logistic regression, random forest, and support vector machine classifiers, and combine their class probabilities using weighted late fusion. The results show that image information is the most effective single modality for publication decade and target readership, whereas text information is the most effective for genre. Multimodal fusion did not improve publication-decade or target-readership estimation, whereas combining text and emotion features improved genre estimation. A genre-level analysis further shows that the most effective modality varies across genres: text is effective for many content-based genres, image information contributes to visually distinctive genres, and emotion information complements text for several narrative genres. These results show the importance of selecting and combining modalities based on tag type and individual genre.Keywords
All Related Versions
This publication has 49 references indexed in Scilit:
- The Publishing and Distribution System of Japanese Manga and DoujinshiPublishing Research Quarterly, 2022
- A multi-label movie genre classification scheme based on the movie’s subtitlesMultimedia Tools and Applications, 2022
- Advances in Collaborative Filtering2021
- CNN-based segmentation of speech balloons and narrative text boxes from comic book page imagesInternational Journal on Document Analysis and Recognition (IJDAR), 2021
- Collaborative Recommendation System in Users of Anime FilmsJournal of Physics: Conference Series, 2020
- AniReco: Japanese Anime Recommendation System2017
- Text-Independent Speech Balloon Segmentation for Comics and Manga2017
- Sketch-based manga retrieval using manga109 datasetMultimedia Tools and Applications, 2016
- The vocabulary of manga: Visual morphology in dialects of Japanese Visual LanguageJournal of Pragmatics, 2016
- Content-based Recommender Systems: State of the Art and Trends2010