THTakatsugu Hirayama

Takatsugu Hirayama

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Takatsugu Hirayama has a total of 78 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Small Object Detection for Birds with Swin Transformer

    by Da Huo, Marc A. Kastner, Tingwei Liu, Yasutomo Kawanishi, Takatsugu Hirayama, Takahiro Komamizu, Ichiro Ide

    Object detection is the task of detecting objects in an image. In this task, the detection of small objects is particularly difficult. Other than the small size, it is also accompanied by difficulties due to blur, occlusion, and so on. Current small object detection methods are tailored to small and dense situations, such as pedestrians in a crowd or far objects in remote sensing scenarios. However, when the target object is small and sparse, there is a lack of objects available for training, making it more difficult to learn effective features. In this paper, we propose a specialized method for detecting a specific category of small objects; birds. Particularly, we improve the features learned by the neck; the sub-network between the backbone and the prediction head, to learn more effective features with a hierarchical design. We employ Swin Transformer to upsample the image features. Moreover, we change the shifted window size for adapting to small objects. Experiments show that the proposed Swin Transformer-based neck combined with CenterNet can lead to good performance by changing the window sizes. We further find that smaller window sizes (default 2) benefit mAPs for small object detection.

  • Open Access

    Semantic Alignment on Action for Image Captioning

    by Da Huo, Marc A. Kastner, Takatsugu Hirayama, Takahiro Komamizu, Yasutomo Kawanishi, Ichiro Ide

    Image captioning is a popular task in vision and language processing, which aims to generate textual descriptions for images. Previously, it simply used image and text as input with self-attention to capture global dependencies. Recent research further uses objects detected from the input image, so-called object tags, as anchor points to ease alignment between image and text with the attention mechanism. However, they only consider object information in images, while neglecting the actions and object interactions that also appear in the image, which causes actions not caught properly in image captioning. To tackle this previously underrepresented dimension of the semantic alignment, we take account of actions on the semantic level. Specifically, our work focuses on human actions and interactions, which ensures that more salient parts of the image get captioned. We introduce a new type of tag, called action tag, to anchor the action information. First, we provide a method for obtaining such action tags using an action detection model which predicts actions in the image. Next, we leverage these action tags into the captioning model. Experimental results indicate that the proposed action tags can help learn action semantics and catch the salient actions leading to perceived improvements in common performance. Experimental results on MS-COCO Karpathy test split show that the proposed model achieves good scores in BLEU-4 and CIDEr metrics, using action tags as anchors. Furthermore, the number of action tags (no more than 5) is smaller than that of object tags (commonly more than 20), which means there is a potential to reduce FLOPs by reducing the total sequence length. It indicates the potential for efficient reasoning and may be applied to daily activity scenes in the future.

  • Origami Crease Recognition for Automatic Folding Diagrams Generation

    by Hitomi Kato, Hirotaka Kato, Takatsugu Hirayama, Takahiro Komamizu, Ichiro Ide

  • Lip Shape-Aware Word Selection for Lyric Translation

    by Kotaro Ikeda, Chihaya Matsuhira, Hirotaka Kato, Marc A. Kastner, Takatsugu Hirayama, Takahiro Komamizu, Ichiro Ide

  • Open Access

    Pre-Instruction for Pedestrians Interacting Autonomous Vehicles With eHMI: Effects on Their Psychology and Walking Behavior

    by Hailong Liu, Takatsugu Hirayama

    External human-machine interface (eHMI) is considered as a new explicit communication method for pedestrian-AV interactions, particularly in encounter scenarios. Pedestrians without prior negotiation experience with eHMI may misinterpret the driving intentions of the AV, leading to confusion and unpredictable behavior. To address this issue, this study suggests providing pre-instruction on eHMI to enhance comprehension. To compare pedestrians’ subjective feelings and walking behavior changes with and without the use of eHMI, as well as before and after receiving pre-instructions, a road crossing experiment using a within-subject design was conducted. In the experiment, the participants were challenged to recognize situations and experienced uncertainty when encountering AVs lacking eHMI, in contrast to manual driving vehicles. After the pre-instruction, participants could understand the driving intention of an AV with eHMI and predict its driving behavior more easily. Furthermore, participants’ subjective feelings and hesitation to make decisions improved to align with the same criteria as encountered with a manual driving vehicle. Additionally, this study found that the information guidance effect of using eHMI made participants’ walking speeds more consistent over multiple trials, as they gain a comprehensive understanding of eHMI principles through the pre-instruction.


  • Quantifying Image-Adjective Associations by Leveraging Large-Scale Pretrained Models

    by Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Ichiro Ide

  • Open Access

    Investigating Conceptual Blending of a Diffusion Model for Improving Nonword-to-Image Generation

    by Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Ichiro Ide

    Text-to-image diffusion models sometimes depict blended concepts in the generated images. One promising use case of this effect would be the nonword-to-image generation task which attempts to generate images intuitively imaginable from a non-existing word (nonword). To realize nonword-to-image generation, an existing study focused on associating nonwords with similar-sounding words. Since each nonword can have multiple similar-sounding words, generating images containing their blended concepts would increase intuitiveness, facilitating creative activities and promoting computational psycholinguistics. Nevertheless, no existing study has quantitatively evaluated this effect in either diffusion models or the nonword-to-image generation paradigm. Therefore, this paper first analyzes the conceptual blending in a pretrained diffusion model, Stable Diffusion. The analysis reveals that a high percentage of generated images depict blended concepts when inputting an embedding interpolating between the text embeddings of two text prompts referring to different concepts. Next, this paper explores the best text embedding space conversion method of an existing nonword-to-image generation framework to ensure both the occurrence of conceptual blending and image generation quality. We compare the conventional direct prediction approach with the proposed method that combines k-nearest neighbor search and linear regression. Evaluation reveals that the enhanced accuracy of the embedding space conversion by the proposed method improves the image generation quality, while the emergence of conceptual blending could be attributed mainly to the specific dimensions of the high-dimensional text embedding space.


  • Open Access

    Interpolating the Text-to-Image Correspondence Based on Phonetic and Phonological Similarities for Nonword-to-Image Generation

    by Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide

    Text-to-Image (T2I) generation is the task of synthesizing images corresponding to a given text input. The recent innovations in artificial intelligence have enhanced the capacity of conventional T2I generation, yielding more and more powerful models day by day. However, their behavior is known to become unstable in the face of text inputs containing nonwords that have no definition within a language. This behavior not only results in situations where image generation does not match human expectations but also hinders these models from being utilized in psycholinguistic applications and simulations. This paper exploits the human nature of associating nonwords with their phonetically and phonologically similar words and uses it to propose a T2I generation framework robust against nonword inputs. The framework comprises a phonetics-aware language model as well as an adjusted T2I generation model. Our evaluations confirm that the proposed nonword-to-image generation synthesizes images that depict visual concepts of phonetically similar words more stably than comparative methods. We also assess how the image generation results match human expectations, showing a better agreement than the phonetics-blind baseline.


  • Open Access

    Correction to: Computational measurement of perceived pointiness from pronunciation

    by Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Ichiro Ide, Takatsugu Hirayama, Yasutomo Kawanishi, Keisuke Doman, Daisuke Deguchi