Takashi Nose
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Takashi Nose has a total of 140 co-authors. Here are the top co-authors by number of shared publications:
Akinori Ito
National Institute of Technology
50 shared publicationsYuya Chiba
NTT Communication Science Laboratories,Kyoto,Japan
37 shared publications- Akinori Ito
Graduate School of Engineering, Tohoku University
33 shared publications Takao Kobayashi
26 shared publications- Takao Kobayashi
Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology, Japan
21 shared publications Tomoki Koriyama
15 shared publications
Latest Publications
- Open Access
- 27 October 2026
- Published by MDPI AG in Preprints.org
Multimodal Manga Tag Estimation Using Image, Text, and Emotion Features
Manga conveys its content through multiple modalities, including page images, dialogue, and emotional progression throughout a story. This study investigates how image, text, and emotion information contribute to volume-level manga tag estimation and whether combining these modalities improves estimation performance. We estimate three tag types–publication decade, target readership, and genre–for 109 volumes in the Manga109 dataset. We extract image features from two-page spread images using CLIP, derive text features from the lines contained in each spread, and obtain emotion features from valence–arousal emotion arcs generated by a large multimodal model. We examine logistic regression, random forest, and support vector machine classifiers, and combine their class probabilities using weighted late fusion. The results show that image information is the most effective single modality for publication decade and target readership, whereas text information is the most effective for genre. Multimodal fusion did not improve publication-decade or target-readership estimation, whereas combining text and emotion features improved genre estimation. A genre-level analysis further shows that the most effective modality varies across genres: text is effective for many content-based genres, image information contributes to visually distinctive genres, and emotion information complements text for several narrative genres. These results show the importance of selecting and combining modalities based on tag type and individual genre.
- 27 October 2026
- Published by MDPI AG in Preprints.org
Open AccessMultimodal Manga Tag Estimation Using Image, Text, and Emotion Features
Manga conveys its content through multiple modalities, including page images, dialogue, and emotional progression throughout a story. This study investigates how image, text, and emotion information contribute to volume-level manga tag estimation and whether combining these modalities improves estimation performance. We estimate three tag types–publication decade, target readership, and genre–for 109 volumes in the Manga109 dataset. We extract image features from two-page spread images using CLIP, derive text features from the lines contained in each spread, and obtain emotion features from valence–arousal emotion arcs generated by a large multimodal model. We examine logistic regression, random forest, and support vector machine classifiers, and combine their class probabilities using weighted late fusion. The results show that image information is the most effective single modality for publication decade and target readership, whereas text information is the most effective for genre. Multimodal fusion did not improve publication-decade or target-readership estimation, whereas combining text and emotion features improved genre estimation. A genre-level analysis further shows that the most effective modality varies across genres: text is effective for many content-based genres, image information contributes to visually distinctive genres, and emotion information complements text for several narrative genres. These results show the importance of selecting and combining modalities based on tag type and individual genre.
- Open Access
- 25 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 50191-50203
Controllable End-to-End Neural Source–Filter Vocoder With Low-Dimensional Speech Parameters and Data Augmentation
Recent neural vocoders synthesize speech waveforms from high-dimensional spectrograms, making it difficult to control individual speech parameters such as F0 and formants. Previous research has proposed a speech synthesis system that controls speech parameters by converting low-dimensional parameters into spectrograms that serve as input to a neural vocoder. However, the quality of the synthesized speech degrades because the generated spectral time series differs from the natural speech used to train the neural vocoder. In this study, we propose a controllable end-to-end neural vocoder that directly synthesizes speech waveforms from low-dimensional speech parameters. Furthermore, we aim to improve the synthesis quality during speech-parameter control by introducing a source–filter model and employing data augmentation. Experimental evaluations show that the source-filter-based method synthesizes speech with quality equal to or better than that of conventional approaches when controlling speech parameters. Additionally, data augmentation improved the robustness of F0 and formant controls. A demonstration of the proposed methods is available online (https://misumisumi.github.io/myprojects/end-to-end-neural-formant-synthesis).
- 25 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 50191-50203
Open AccessControllable End-to-End Neural Source–Filter Vocoder With Low-Dimensional Speech Parameters and Data Augmentation
Recent neural vocoders synthesize speech waveforms from high-dimensional spectrograms, making it difficult to control individual speech parameters such as F0 and formants. Previous research has proposed a speech synthesis system that controls speech parameters by converting low-dimensional parameters into spectrograms that serve as input to a neural vocoder. However, the quality of the synthesized speech degrades because the generated spectral time series differs from the natural speech used to train the neural vocoder. In this study, we propose a controllable end-to-end neural vocoder that directly synthesizes speech waveforms from low-dimensional speech parameters. Furthermore, we aim to improve the synthesis quality during speech-parameter control by introducing a source–filter model and employing data augmentation. Experimental evaluations show that the source-filter-based method synthesizes speech with quality equal to or better than that of conventional approaches when controlling speech parameters. Additionally, data augmentation improved the robustness of F0 and formant controls. A demonstration of the proposed methods is available online (https://misumisumi.github.io/myprojects/end-to-end-neural-formant-synthesis).
- Open Access
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 162756-162771
Stable and High-Fidelity Photo-Realistic Talking Face Generation Using a Three-Stage Model With Extended Landmarks and Post-Refinement
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 162756-162771
Open AccessStable and High-Fidelity Photo-Realistic Talking Face Generation Using a Three-Stage Model With Extended Landmarks and Post-Refinement
- Open Access
- 3 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. PP, 1
Adaptive Fine-Grained Pruning via Binary Search for Efficient Environmental Sound Classification
The overparameterization of pretrained transformers limits their efficiency in Environmental Sound Classification (ESC), incurring high memory and computational overhead. Pruning redundant parameters is a common remedy, yet fine-grained pruning at the intra-layer level remains difficult due to the large combinatorial search space and the risk of accuracy degradation. In this paper, we propose Adaptive Fine-Grained Pruning (AFP), an adaptive pruning method operating at intra-layer granularity under a user-defined performance constraint. AFP estimates the importance of prunable components by accumulating gradients during fine-tuning and then performs a binary search over the importance-ranked substructure space in a coarse-to-fine manner to identify the smallest submodel that satisfies the constraint. AFP avoids post-pruning gradient recalibration and efficiently identifies the smallest submodel via binary search, without traversing the full substructure space. Experiments show that AFP adaptively achieves up to 66.56% and 45.84% parameter reductions on the standard Audio Spectrogram Transformer for ESC-50 and UrbanSound8K, respectively, within a 2% drop in accuracy, outperforming popular baselines of static pruning or layer-level pruning in both pruning effectiveness and search efficiency.
- 3 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. PP, 1
Open AccessAdaptive Fine-Grained Pruning via Binary Search for Efficient Environmental Sound Classification
The overparameterization of pretrained transformers limits their efficiency in Environmental Sound Classification (ESC), incurring high memory and computational overhead. Pruning redundant parameters is a common remedy, yet fine-grained pruning at the intra-layer level remains difficult due to the large combinatorial search space and the risk of accuracy degradation. In this paper, we propose Adaptive Fine-Grained Pruning (AFP), an adaptive pruning method operating at intra-layer granularity under a user-defined performance constraint. AFP estimates the importance of prunable components by accumulating gradients during fine-tuning and then performs a binary search over the importance-ranked substructure space in a coarse-to-fine manner to identify the smallest submodel that satisfies the constraint. AFP avoids post-pruning gradient recalibration and efficiently identifies the smallest submodel via binary search, without traversing the full substructure space. Experiments show that AFP adaptively achieves up to 66.56% and 45.84% parameter reductions on the standard Audio Spectrogram Transformer for ESC-50 and UrbanSound8K, respectively, within a 2% drop in accuracy, outperforming popular baselines of static pruning or layer-level pruning in both pruning effectiveness and search efficiency.
- 1 October 2025
- Published by Springer Nature
- p. 103-112
Generation of Listening Motion of Embodied Conversational Agents Using Speech and Text Information
- 1 October 2025
- Published by Springer Nature
- p. 103-112
Generation of Listening Motion of Embodied Conversational Agents Using Speech and Text Information
- 18 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 177-181
Japanese Shadowing Training Using Synchronized Partial Captions
Shadowing is a language practice method where a learner listens to a native speaker's speech and speaks the same content immediately. Speech shadowing is a popular languagelearning technique that is effective for developing listening and speaking skills. This paper proposes using synchronized partial captions (PSC) in the shadowing training. In PSC, the text is synchronized to the speech at the word level, and only selected words are shown in the caption to aid listening comprehension, while the rest are masked to keep the learner listening to the speech. We conducted an experiment where 59 learners exercised shadowing in one of three learning methods: no caption, full caption, and partial caption. The experimental result shows that training in PSC conditions rapidly improves learners' proficiency.
- 18 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 177-181
Japanese Shadowing Training Using Synchronized Partial Captions
Shadowing is a language practice method where a learner listens to a native speaker's speech and speaks the same content immediately. Speech shadowing is a popular languagelearning technique that is effective for developing listening and speaking skills. This paper proposes using synchronized partial captions (PSC) in the shadowing training. In PSC, the text is synchronized to the speech at the word level, and only selected words are shown in the caption to aid listening comprehension, while the rest are masked to keep the learner listening to the speech. We conducted an experiment where 59 learners exercised shadowing in one of three learning methods: no caption, full caption, and partial caption. The experimental result shows that training in PSC conditions rapidly improves learners' proficiency.
- Open Access
- 11 April 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15  (8) , 4256
The Development of an Emotional Embodied Conversational Agent and the Evaluation of the Effect of Response Delay on User Impression
Embodied conversational agents (ECAs) are autonomous interaction interfaces designed to communicate with humans. This study investigates the impact of response delays and emotional facial expressions of ECAs on user perception and engagement. The motivation for this study stems from the growing integration of ECAs in various sectors, where their ability to mimic human-like interactions significantly enhances user experience. To this end, we developed an ECA with multimodal emotion recognition, both with voice and facial feature recognition and emotional facial expressions of the agent avatar. The system generates answers in real time based on media content. The development was supported by a case study of artwork images with the agent playing the role of a museum curator, where the user asks the agent for information on the artwork. We evaluated the developed system in two aspects. First, we investigated how the delay in an agent’s responses influences user satisfaction and perception. Secondly, we explored the role of emotion in an ECA’s face in shaping the user’s perception of responsiveness. The results showed that the longer response delay negatively impacted the user’s perception of responsiveness when the ECA did not express emotion, while the emotional expression improved the responsiveness perception.
- 11 April 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15  (8) , 4256
Open AccessThe Development of an Emotional Embodied Conversational Agent and the Evaluation of the Effect of Response Delay on User Impression
Embodied conversational agents (ECAs) are autonomous interaction interfaces designed to communicate with humans. This study investigates the impact of response delays and emotional facial expressions of ECAs on user perception and engagement. The motivation for this study stems from the growing integration of ECAs in various sectors, where their ability to mimic human-like interactions significantly enhances user experience. To this end, we developed an ECA with multimodal emotion recognition, both with voice and facial feature recognition and emotional facial expressions of the agent avatar. The system generates answers in real time based on media content. The development was supported by a case study of artwork images with the agent playing the role of a museum curator, where the user asks the agent for information on the artwork. We evaluated the developed system in two aspects. First, we investigated how the delay in an agent’s responses influences user satisfaction and perception. Secondly, we explored the role of emotion in an ECA’s face in shaping the user’s perception of responsiveness. The results showed that the longer response delay negatively impacted the user’s perception of responsiveness when the ECA did not express emotion, while the emotional expression improved the responsiveness perception.
- Open Access
- 1 January 2025
- Published by Acoustical Society of Japan in Acoustical Science and Technology
- Vol. 46  (1) , 116-119
Fast end-to-end non-parallel voice conversion based on speaker-adaptive neural vocoder with cycle-consistent learning
This paper proposes a fast end-to-end non-parallel voice conversion (VC) named Tachylone. In Thachylone, speaker conversion and waveform generation is performed by a single vocoder network. In the training of Tachylone, a pre-trained universal neural vocoder is used as the initial model, and the model parameters are updated using source and target speakers’ non-parallel data based on cycle-consistent learning in an end-to-end manner. We compare Tachylone to conventional CycleGAN-based VC with objective and subjective measures and discuss the results.
- 1 January 2025
- Published by Acoustical Society of Japan in Acoustical Science and Technology
- Vol. 46  (1) , 116-119
Open AccessFast end-to-end non-parallel voice conversion based on speaker-adaptive neural vocoder with cycle-consistent learning
This paper proposes a fast end-to-end non-parallel voice conversion (VC) named Tachylone. In Thachylone, speaker conversion and waveform generation is performed by a single vocoder network. In the training of Tachylone, a pre-trained universal neural vocoder is used as the initial model, and the model parameters are updated using source and target speakers’ non-parallel data based on cycle-consistent learning in an end-to-end manner. We compare Tachylone to conventional CycleGAN-based VC with objective and subjective measures and discuss the results.
- Open Access
- 1 January 2025
- Published by Acoustical Society of Japan in Acoustical Science and Technology
- Vol. 46  (1) , 120-123
Unified model for voice conversion of speech and singing voice using adaptive pitch constraints
This paper proposes a voice conversion named SpSiVC that appropriately converts both speech and singing voices with a single model. Since the distribution of pitch between speakers is significantly different for speech and singing voices, voice conversion has been mainly evaluated as a separate task for speech and singing voice conversion. SpSiVC introduces an adaptive F0 loss, which enables conversion that implicitly switches the shift width of the logarithm F0 according to the type of input voice. We examine the effectiveness of the F0 constraints in objective and subjective evaluations.
- 1 January 2025
- Published by Acoustical Society of Japan in Acoustical Science and Technology
- Vol. 46  (1) , 120-123
Open AccessUnified model for voice conversion of speech and singing voice using adaptive pitch constraints
This paper proposes a voice conversion named SpSiVC that appropriately converts both speech and singing voices with a single model. Since the distribution of pitch between speakers is significantly different for speech and singing voices, voice conversion has been mainly evaluated as a separate task for speech and singing voice conversion. SpSiVC introduces an adaptive F0 loss, which enables conversion that implicitly switches the shift width of the logarithm F0 according to the type of input voice. We examine the effectiveness of the F0 constraints in objective and subjective evaluations.
- Open Access
- 13 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 369-373
Estimation of Offensiveness of Posts on Social Media and Its Application to a Conversation Assistance System
- 13 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 369-373
Open AccessEstimation of Offensiveness of Posts on Social Media and Its Application to a Conversation Assistance System