Akinori Ito
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Akinori Ito has a total of 140 co-authors. Here are the top co-authors by number of shared publications:
- Takashi Nose
Graduate School of Engineering, Tohoku University, Sendai, Japan
33 shared publications Shozo Makino
Tohoku Bunka Gakuen University, Sendai, Japan
23 shared publicationsYuya Chiba
NTT Communication Sciences Laboratory
16 shared publicationsMasashi Ito
13 shared publicationsMotoyuki Suzuki
Institute of Science and Technology, The University of Tokushima, Japan
12 shared publications- Yutaka Hiroi
Faculty of Robotics and Design, Osaka Institute of Technology, Osaka 530-8568, Japan
12 shared publications
Latest Publications
- Open Access
- 27 October 2026
- Published by MDPI AG in Preprints.org
Multimodal Manga Tag Estimation Using Image, Text, and Emotion Features
Manga conveys its content through multiple modalities, including page images, dialogue, and emotional progression throughout a story. This study investigates how image, text, and emotion information contribute to volume-level manga tag estimation and whether combining these modalities improves estimation performance. We estimate three tag types–publication decade, target readership, and genre–for 109 volumes in the Manga109 dataset. We extract image features from two-page spread images using CLIP, derive text features from the lines contained in each spread, and obtain emotion features from valence–arousal emotion arcs generated by a large multimodal model. We examine logistic regression, random forest, and support vector machine classifiers, and combine their class probabilities using weighted late fusion. The results show that image information is the most effective single modality for publication decade and target readership, whereas text information is the most effective for genre. Multimodal fusion did not improve publication-decade or target-readership estimation, whereas combining text and emotion features improved genre estimation. A genre-level analysis further shows that the most effective modality varies across genres: text is effective for many content-based genres, image information contributes to visually distinctive genres, and emotion information complements text for several narrative genres. These results show the importance of selecting and combining modalities based on tag type and individual genre.
- 27 October 2026
- Published by MDPI AG in Preprints.org
Open AccessMultimodal Manga Tag Estimation Using Image, Text, and Emotion Features
Manga conveys its content through multiple modalities, including page images, dialogue, and emotional progression throughout a story. This study investigates how image, text, and emotion information contribute to volume-level manga tag estimation and whether combining these modalities improves estimation performance. We estimate three tag types–publication decade, target readership, and genre–for 109 volumes in the Manga109 dataset. We extract image features from two-page spread images using CLIP, derive text features from the lines contained in each spread, and obtain emotion features from valence–arousal emotion arcs generated by a large multimodal model. We examine logistic regression, random forest, and support vector machine classifiers, and combine their class probabilities using weighted late fusion. The results show that image information is the most effective single modality for publication decade and target readership, whereas text information is the most effective for genre. Multimodal fusion did not improve publication-decade or target-readership estimation, whereas combining text and emotion features improved genre estimation. A genre-level analysis further shows that the most effective modality varies across genres: text is effective for many content-based genres, image information contributes to visually distinctive genres, and emotion information complements text for several narrative genres. These results show the importance of selecting and combining modalities based on tag type and individual genre.
- 18 November 2025
- Published by Springer Nature
- p. 247-256
Embedding Digital Signature into CSV Files Using Data Hiding
- 18 November 2025
- Published by Springer Nature
- p. 247-256
Embedding Digital Signature into CSV Files Using Data Hiding
- 1 October 2025
- Published by Springer Nature
- p. 103-112
Generation of Listening Motion of Embodied Conversational Agents Using Speech and Text Information
- 1 October 2025
- Published by Springer Nature
- p. 103-112
Generation of Listening Motion of Embodied Conversational Agents Using Speech and Text Information
- 18 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 177-181
Japanese Shadowing Training Using Synchronized Partial Captions
Shadowing is a language practice method where a learner listens to a native speaker's speech and speaks the same content immediately. Speech shadowing is a popular languagelearning technique that is effective for developing listening and speaking skills. This paper proposes using synchronized partial captions (PSC) in the shadowing training. In PSC, the text is synchronized to the speech at the word level, and only selected words are shown in the caption to aid listening comprehension, while the rest are masked to keep the learner listening to the speech. We conducted an experiment where 59 learners exercised shadowing in one of three learning methods: no caption, full caption, and partial caption. The experimental result shows that training in PSC conditions rapidly improves learners' proficiency.
- 18 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 177-181
Japanese Shadowing Training Using Synchronized Partial Captions
Shadowing is a language practice method where a learner listens to a native speaker's speech and speaks the same content immediately. Speech shadowing is a popular languagelearning technique that is effective for developing listening and speaking skills. This paper proposes using synchronized partial captions (PSC) in the shadowing training. In PSC, the text is synchronized to the speech at the word level, and only selected words are shown in the caption to aid listening comprehension, while the rest are masked to keep the learner listening to the speech. We conducted an experiment where 59 learners exercised shadowing in one of three learning methods: no caption, full caption, and partial caption. The experimental result shows that training in PSC conditions rapidly improves learners' proficiency.
- Open Access
- 11 April 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (8) , 4256
The Development of an Emotional Embodied Conversational Agent and the Evaluation of the Effect of Response Delay on User Impression
Embodied conversational agents (ECAs) are autonomous interaction interfaces designed to communicate with humans. This study investigates the impact of response delays and emotional facial expressions of ECAs on user perception and engagement. The motivation for this study stems from the growing integration of ECAs in various sectors, where their ability to mimic human-like interactions significantly enhances user experience. To this end, we developed an ECA with multimodal emotion recognition, both with voice and facial feature recognition and emotional facial expressions of the agent avatar. The system generates answers in real time based on media content. The development was supported by a case study of artwork images with the agent playing the role of a museum curator, where the user asks the agent for information on the artwork. We evaluated the developed system in two aspects. First, we investigated how the delay in an agent’s responses influences user satisfaction and perception. Secondly, we explored the role of emotion in an ECA’s face in shaping the user’s perception of responsiveness. The results showed that the longer response delay negatively impacted the user’s perception of responsiveness when the ECA did not express emotion, while the emotional expression improved the responsiveness perception.
- 11 April 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (8) , 4256
Open AccessThe Development of an Emotional Embodied Conversational Agent and the Evaluation of the Effect of Response Delay on User Impression
Embodied conversational agents (ECAs) are autonomous interaction interfaces designed to communicate with humans. This study investigates the impact of response delays and emotional facial expressions of ECAs on user perception and engagement. The motivation for this study stems from the growing integration of ECAs in various sectors, where their ability to mimic human-like interactions significantly enhances user experience. To this end, we developed an ECA with multimodal emotion recognition, both with voice and facial feature recognition and emotional facial expressions of the agent avatar. The system generates answers in real time based on media content. The development was supported by a case study of artwork images with the agent playing the role of a museum curator, where the user asks the agent for information on the artwork. We evaluated the developed system in two aspects. First, we investigated how the delay in an agent’s responses influences user satisfaction and perception. Secondly, we explored the role of emotion in an ECA’s face in shaping the user’s perception of responsiveness. The results showed that the longer response delay negatively impacted the user’s perception of responsiveness when the ECA did not express emotion, while the emotional expression improved the responsiveness perception.
- Open Access
Robust Human Tracking Using a 3D LiDAR and Point Cloud Projection for Human-Following Robots
Human tracking is a fundamental technology for mobile robots that work with humans. Various devices are used to observe humans, such as cameras, RGB-D sensors, millimeter-wave radars, and laser range finders (LRF). Typical LRF measurements observe only the surroundings on a particular horizontal plane. Human recognition using an LRF has a low computational load and is suitable for mobile robots. However, it is vulnerable to variations in human height, potentially leading to detection failures for individuals taller or shorter than the standard height. This work aims to develop a method that is robust to height differences among humans using a 3D LiDAR. We observed the environment using a 3D LiDAR and projected the point cloud onto a single horizontal plane to apply a human-tracking method for 2D LRFs. We investigated the optimal height range of the point clouds for projection and found that using 30% of the point clouds from the top of the measured person provided the most stable tracking. The results of the path-following experiments revealed that the proposed method reduced the proportion of outlier points compared to projecting all the points (from 3.63% to 1.75%). As a result, the proposed method was effective in achieving robust human following.
Open AccessRobust Human Tracking Using a 3D LiDAR and Point Cloud Projection for Human-Following Robots
Human tracking is a fundamental technology for mobile robots that work with humans. Various devices are used to observe humans, such as cameras, RGB-D sensors, millimeter-wave radars, and laser range finders (LRF). Typical LRF measurements observe only the surroundings on a particular horizontal plane. Human recognition using an LRF has a low computational load and is suitable for mobile robots. However, it is vulnerable to variations in human height, potentially leading to detection failures for individuals taller or shorter than the standard height. This work aims to develop a method that is robust to height differences among humans using a 3D LiDAR. We observed the environment using a 3D LiDAR and projected the point cloud onto a single horizontal plane to apply a human-tracking method for 2D LRFs. We investigated the optimal height range of the point clouds for projection and found that using 30% of the point clouds from the top of the measured person provided the most stable tracking. The results of the path-following experiments revealed that the proposed method reduced the proportion of outlier points compared to projecting all the points (from 3.63% to 1.75%). As a result, the proposed method was effective in achieving robust human following.
- Open Access
- 3 January 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (1) , 381
Reversible Spectral Speech Watermarking with Variable Embedding Locations Against Spectrum-Based Attacks
To guarantee the reliability and integrity of audio, data have been focused on as an essential topic as the fast development of generative AI. Significant progress in machine learning and speech synthesis has increased the potential for audio tampering. In this paper, we focus on the digital watermarking method as a promising method to safeguard the authenticity of audio evidence. Due to the integrity of the original data with probative importance, the algorithm requires reversibility, imperceptibility, and reliability. To meet the requirements, we propose a reversible digital watermarking approach that embeds feature data concentrating in high-frequency intDCT coefficients after transforming data from the time domain into the frequency domain. We explored the appropriate hiding locations against spectrum-based attacks with novel proposed methodologies for spectral expansion for embedding. However, the drawback of fixed expansion is that the stego signal is prone to being detected by a spectral analysis. Therefore, this paper proposes two other new expansion methodologies that embed the data into variable locations—random expansion and adaptive expansion with distortion estimation for embedding—which effectively conceal the watermark’s presence while maintaining high perceptual quality with an average segSNR better than 21.363 dB and average MOS value better than 4.085. Our experimental results demonstrate the efficacy of our proposed method in both sound quality preservation and log-likelihood value, indicating the absolute discontinuity of the spectrogram after embedding is proposed to evaluate the effectiveness of the proposed reversible spectral expansion watermarking algorithm. The result of EER indicated that the adaptive hiding performed best against attacks by spectral analysis.
- 3 January 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (1) , 381
Open AccessReversible Spectral Speech Watermarking with Variable Embedding Locations Against Spectrum-Based Attacks
To guarantee the reliability and integrity of audio, data have been focused on as an essential topic as the fast development of generative AI. Significant progress in machine learning and speech synthesis has increased the potential for audio tampering. In this paper, we focus on the digital watermarking method as a promising method to safeguard the authenticity of audio evidence. Due to the integrity of the original data with probative importance, the algorithm requires reversibility, imperceptibility, and reliability. To meet the requirements, we propose a reversible digital watermarking approach that embeds feature data concentrating in high-frequency intDCT coefficients after transforming data from the time domain into the frequency domain. We explored the appropriate hiding locations against spectrum-based attacks with novel proposed methodologies for spectral expansion for embedding. However, the drawback of fixed expansion is that the stego signal is prone to being detected by a spectral analysis. Therefore, this paper proposes two other new expansion methodologies that embed the data into variable locations—random expansion and adaptive expansion with distortion estimation for embedding—which effectively conceal the watermark’s presence while maintaining high perceptual quality with an average segSNR better than 21.363 dB and average MOS value better than 4.085. Our experimental results demonstrate the efficacy of our proposed method in both sound quality preservation and log-likelihood value, indicating the absolute discontinuity of the spectrogram after embedding is proposed to evaluate the effectiveness of the proposed reversible spectral expansion watermarking algorithm. The result of EER indicated that the adaptive hiding performed best against attacks by spectral analysis.
- Open Access
- 1 January 2025
- Published by Acoustical Society of Japan in Acoustical Science and Technology
- Vol. 46 (1) , 96-99
We open our mouths when we are silent
Lip syncing is an important technology that enhances the impression of embodied conversational agents. However, there is no study to design the mouth movement of the agent when the agent is silent. Therefore, this paper investigated how human speakers move their mouths when silent in dialogues. As a result, we found three facts. First, a speaker does not completely close mouth even when listening to a partner’s talk. Second, the degree of mouth opening while talking and listening greatly depends on the speaker. Third, the mouth opening is possibly affected by the next state of the speaker.
- 1 January 2025
- Published by Acoustical Society of Japan in Acoustical Science and Technology
- Vol. 46 (1) , 96-99
Open AccessWe open our mouths when we are silent
Lip syncing is an important technology that enhances the impression of embodied conversational agents. However, there is no study to design the mouth movement of the agent when the agent is silent. Therefore, this paper investigated how human speakers move their mouths when silent in dialogues. As a result, we found three facts. First, a speaker does not completely close mouth even when listening to a partner’s talk. Second, the degree of mouth opening while talking and listening greatly depends on the speaker. Third, the mouth opening is possibly affected by the next state of the speaker.
- Open Access
- 13 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 369-373
Estimation of Offensiveness of Posts on Social Media and Its Application to a Conversation Assistance System
- 13 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 369-373
Open AccessEstimation of Offensiveness of Posts on Social Media and Its Application to a Conversation Assistance System
- Open Access
- 27 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. PP, 1
Speaker Intimacy Estimation in Chat-Talks Based on Verbal and Non-Verbal Information
Conversations based on mutual intimacy are critical for maintaining positive relationships. A detailed understanding of speaker relationships in dialogues enhances various applications, such as information recommendation systems. Such systems, when interacting with multiple users, can provide more tailored information by understanding the users’ relationships. Furthermore, dialogue systems, which are becoming increasingly prevalent in society, can foster long-term user engagement by recognizing and responding to the intimacy levels of users. This study explores a method for estimating the intimacy levels of speakers and dialogue partners in conversational exchanges. Our approach utilizes a multimodal corpus of natural conversations with 71 Japanese participants, complete with metadata indicating each speaker’s perceived intimacy level. We identified key features for estimating intimacy by analyzing the statistical parameters of these features. Our comprehensive analysis encompassed both verbal and non-verbal information, including prosody, gestures, and facial expressions. The proposed intimacy estimation model combines multimodal features using a multi-stream Bi-directional Long Short-Term Memory (BLSTM) network and grasps the contextual information of conversations with a Context BLSTM. Our model’s effectiveness is demonstrated through comparisons with several baseline models. Experimental results show that our proposed model significantly improves the overall performance compared with other models. Although the RoBERTa-based method (the best baseline model) achieved an F1 score of 0.571, our method had an F1 score of 0.594. In particular, an ablation study shows that combining verbal and non-verbal features is useful for intimacy estimation. The performance was further improved by extending the dialogue context, showing that the proposed model can estimate three levels of intimacy with an F1 score of 0.666 by observing eight utterance exchanges.
- 27 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. PP, 1
Open AccessSpeaker Intimacy Estimation in Chat-Talks Based on Verbal and Non-Verbal Information
Conversations based on mutual intimacy are critical for maintaining positive relationships. A detailed understanding of speaker relationships in dialogues enhances various applications, such as information recommendation systems. Such systems, when interacting with multiple users, can provide more tailored information by understanding the users’ relationships. Furthermore, dialogue systems, which are becoming increasingly prevalent in society, can foster long-term user engagement by recognizing and responding to the intimacy levels of users. This study explores a method for estimating the intimacy levels of speakers and dialogue partners in conversational exchanges. Our approach utilizes a multimodal corpus of natural conversations with 71 Japanese participants, complete with metadata indicating each speaker’s perceived intimacy level. We identified key features for estimating intimacy by analyzing the statistical parameters of these features. Our comprehensive analysis encompassed both verbal and non-verbal information, including prosody, gestures, and facial expressions. The proposed intimacy estimation model combines multimodal features using a multi-stream Bi-directional Long Short-Term Memory (BLSTM) network and grasps the contextual information of conversations with a Context BLSTM. Our model’s effectiveness is demonstrated through comparisons with several baseline models. Experimental results show that our proposed model significantly improves the overall performance compared with other models. Although the RoBERTa-based method (the best baseline model) achieved an F1 score of 0.571, our method had an F1 score of 0.594. In particular, an ablation study shows that combining verbal and non-verbal features is useful for intimacy estimation. The performance was further improved by extending the dialogue context, showing that the proposed model can estimate three levels of intimacy with an F1 score of 0.666 by observing eight utterance exchanges.