Youngwan Jin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Youngwan Jin has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Hyeongjin Ju
School of Integrated Technology, Yonsei University, Incheon, Republic of Korea
5 shared publications - Incheol Park
School of Integrated Technology, Yonsei University, Incheon, Republic of Korea
5 shared publications - Sanghyeop Yeo
School of Integrated Technology, Yonsei University, Incheon, Republic of Korea
5 shared publications - Shiho Kim
Yonsei University, Seoul, Republic of Korea
5 shared publications - Yagiz Nalcakan
School of Integrated Technology, Yonsei University, Incheon 21983, Republic of Korea
5 shared publications - Michal Kovac
Slovak University of Technology in Bratislava
1 shared publication
Latest Publications
- Open Access
- 7 August 2026
- Published by Elsevier in Results in Engineering
- Vol. 32
RM-MLLM: Robust scene understanding in adverse weather with RGB-multispectral fusion
- 7 August 2026
- Published by Elsevier in Results in Engineering
- Vol. 32
Open AccessRM-MLLM: Robust scene understanding in adverse weather with RGB-multispectral fusion
- Open Access
- 7 May 2026
- Published by Elsevier in SSRN Electronic Journal
RM-MLLM: Robust Scene Understanding in Adverse Weather with RGB-Multispectral Fusion
Multimodal large language models (MLLMs) for autonomous driving often become less reliable in adverse weather, partly because training data are dominated by clear, well-lit scenes, and most models primarily rely on RGB imagery. We address these issues with two contributions. First, we introduce DENSE-QA, a large-scale benchmark for adverse-weather scene understanding comprising 12,997 scenes and over 77,000 instruction-response pairs, built on synchronized RGB and multispectral imagery. Second, we propose RM-MLLM, an MLLM architecture with modality-specific encoders and an adaptive fusion module, trained with a three-stage progressive strategy to better integrate complementary sensor cues. Across multiple LLM backbones, RM-MLLM improves QA performance under adverse conditions. Using only RGB input, the proposed architecture and training already outperform all evaluated baselines, including prompt-augmented models, by over 11 percentage points. Incorporating RGB-multispectral fusion provides an additional 3-point gain concentrated on weather-sensitive tasks. We also observe that fine-tuning a strong RGB-only vision-language model on the same training data yields performance comparable to its zero-shot evaluation, suggesting that DENSE-QA fine-tuning alone does not close the gap to RM-MLLM and that multispectral input provides complementary value. We release the benchmark and models to support future all-weather autonomous driving research https://yonsei-stl.github.io/RM-MLLM.
- 7 May 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessRM-MLLM: Robust Scene Understanding in Adverse Weather with RGB-Multispectral Fusion
Multimodal large language models (MLLMs) for autonomous driving often become less reliable in adverse weather, partly because training data are dominated by clear, well-lit scenes, and most models primarily rely on RGB imagery. We address these issues with two contributions. First, we introduce DENSE-QA, a large-scale benchmark for adverse-weather scene understanding comprising 12,997 scenes and over 77,000 instruction-response pairs, built on synchronized RGB and multispectral imagery. Second, we propose RM-MLLM, an MLLM architecture with modality-specific encoders and an adaptive fusion module, trained with a three-stage progressive strategy to better integrate complementary sensor cues. Across multiple LLM backbones, RM-MLLM improves QA performance under adverse conditions. Using only RGB input, the proposed architecture and training already outperform all evaluated baselines, including prompt-augmented models, by over 11 percentage points. Incorporating RGB-multispectral fusion provides an additional 3-point gain concentrated on weather-sensitive tasks. We also observe that fine-tuning a strong RGB-only vision-language model on the same training data yields performance comparable to its zero-shot evaluation, suggesting that DENSE-QA fine-tuning alone does not close the gap to RM-MLLM and that multispectral input provides complementary value. We release the benchmark and models to support future all-weather autonomous driving research https://yonsei-stl.github.io/RM-MLLM.
- 26 February 2026
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 266
SpectraDiff: Enhancing the fidelity of Infrared Image Translation with object-aware diffusion
- 26 February 2026
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 266
SpectraDiff: Enhancing the fidelity of Infrared Image Translation with object-aware diffusion
- Open Access
- 17 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 215882-215896
Multi-Modal Cross-Attention-Based Multi-Label Weather and Severity Level Classification
Ensuring traffic safety in autonomous vehicles requires accurate weather and severity recognition, especially when operating within a defined operational design domain (ODD). However, most existing studies focus on single-label weather recognition and largely overlook weather severity classification. This gap in severity annotation limits the adaptability of autonomous systems under varied weather conditions. Current datasets are typically single-labeled and lack severity annotations, which are crucial for autonomous driving systems to adjust appropriately based on weather intensity. To address this data scarcity, we created a comprehensive multi-label weather dataset with light, moderate, and heavy severity levels based on quantitative visibility thresholds aligned with operational design domains (ODDs). We constructed this dataset by prompting multiple vision-language models (VLMs) to apply these criteria, then validated annotation quality against a human-labeled gold standard. This approach overcomes the limitations of existing datasets. On this new dataset, we introduce the Multi-Weather and Severity Classifier (MWSC), a novel multi-modal framework that simultaneously handles weather recognition and severity classification. MWSC employs a combination of cross-attention mechanisms to extract global and local image features and integrates them with text captions generated from a standardized, closed-vocabulary template for weather and severity to create robust feature associations for classifier training. Our experiments establish a new benchmark for environment perception in autonomous driving, where MWSC significantly outperforms widely-used baseline models across key metrics including mAP, accuracy, and F1-score, advancing vision-based techniques for safer and more reliable autonomous navigation in complex weather scenarios. The full dataset and code are publicly available in our repository at https://github.com/Yonsei-STL/MWSC.git, including licenses of all sources, human-annotated gold set, image lists and splits, prompts, and trained weights for reproducibility.
- 17 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 215882-215896
Open AccessMulti-Modal Cross-Attention-Based Multi-Label Weather and Severity Level Classification
Ensuring traffic safety in autonomous vehicles requires accurate weather and severity recognition, especially when operating within a defined operational design domain (ODD). However, most existing studies focus on single-label weather recognition and largely overlook weather severity classification. This gap in severity annotation limits the adaptability of autonomous systems under varied weather conditions. Current datasets are typically single-labeled and lack severity annotations, which are crucial for autonomous driving systems to adjust appropriately based on weather intensity. To address this data scarcity, we created a comprehensive multi-label weather dataset with light, moderate, and heavy severity levels based on quantitative visibility thresholds aligned with operational design domains (ODDs). We constructed this dataset by prompting multiple vision-language models (VLMs) to apply these criteria, then validated annotation quality against a human-labeled gold standard. This approach overcomes the limitations of existing datasets. On this new dataset, we introduce the Multi-Weather and Severity Classifier (MWSC), a novel multi-modal framework that simultaneously handles weather recognition and severity classification. MWSC employs a combination of cross-attention mechanisms to extract global and local image features and integrates them with text captions generated from a standardized, closed-vocabulary template for weather and severity to create robust feature associations for classifier training. Our experiments establish a new benchmark for environment perception in autonomous driving, where MWSC significantly outperforms widely-used baseline models across key metrics including mAP, accuracy, and F1-score, advancing vision-based techniques for safer and more reliable autonomous navigation in complex weather scenarios. The full dataset and code are publicly available in our repository at https://github.com/Yonsei-STL/MWSC.git, including licenses of all sources, human-annotated gold set, image lists and splits, prompts, and trained weights for reproducibility.
- 29 October 2025
- Published by Elsevier in Information Fusion
- Vol. 128
RASMD: RGB and SWIR multispectral driving dataset for robust perception in adverse conditions
- 29 October 2025
- Published by Elsevier in Information Fusion
- Vol. 128
RASMD: RGB and SWIR multispectral driving dataset for robust perception in adverse conditions