Donghyeok Jo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Donghyeok Jo has a total of 3 co-authors. Here are the top co-authors by number of shared publications:
- Chee Sun Won
Department of Electronics and Electrical Engineering, Dongguk University, Seoul, South Korea
1 shared publication - Jun-Hwa Kim
Department of Artificial Intelligence, Konyang University, Daejeon 35365, Republic of Korea
1 shared publication - Namho Kim
Department of Electronics and Electrical Engineering Dongguk University Seoul Korea
1 shared publication
Latest Publications
- Open Access
- 20 November 2024
- Published by MDPI AG in Electronics
- Vol. 13 (22) , 4552
Multimodal Food Image Classification with Large Language Models
In this study, we leverage advancements in large language models (LLMs) for fine-grained food image classification. We achieve this by integrating textual features extracted from images using an LLM into a multimodal learning framework. Specifically, semantic textual descriptions generated by the LLM are encoded and combined with image features obtained from a transformer-based architecture to improve food image classification. Our approach employs a cross-attention mechanism to effectively fuse visual and textual modalities, enhancing the model’s ability to extract discriminative features beyond what can be achieved with visual features alone.
- 20 November 2024
- Published by MDPI AG in Electronics
- Vol. 13 (22) , 4552
Open AccessMultimodal Food Image Classification with Large Language Models
In this study, we leverage advancements in large language models (LLMs) for fine-grained food image classification. We achieve this by integrating textual features extracted from images using an LLM into a multimodal learning framework. Specifically, semantic textual descriptions generated by the LLM are encoded and combined with image features obtained from a transformer-based architecture to improve food image classification. Our approach employs a cross-attention mechanism to effectively fuse visual and textual modalities, enhancing the model’s ability to extract discriminative features beyond what can be achieved with visual features alone.