Hyeongjin Ju
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Hyeongjin Ju has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Incheol Park
School of Integrated Technology, Yonsei University, Incheon, Republic of Korea
6 shared publications - Sanghyeop Yeo
School of Integrated Technology, Yonsei University, Incheon, Republic of Korea
6 shared publications - Shiho Kim
Yonsei University, Seoul, Republic of Korea
6 shared publications - Yagiz Nalcakan
School of Integrated Technology, Yonsei University, Incheon 21983, Republic of Korea
6 shared publications - Youngwan Jin
School of Integrated Technology, Yonsei University, Incheon, Republic of Korea
5 shared publications - Michal Kovac
Slovak University of Technology in Bratislava
1 shared publication
Latest Publications
- Open Access
- 8 September 2026
- Published by Elsevier in SSRN Electronic Journal
Physics-Guided Multispectral SWIR-Range Perception under Raw-Sensor Domain Shift
Supervised detectors may perform well on curated multispectral data yet behave differently on raw sensor streams. We investigate this gap using a vehicle-mounted four-band short-wave infrared (SWIR) range sensor, an annotated, processed collection, and 35 separately acquired nighttime sequences that contain road objects and controlled wet- and ice-like road-surface surrogates. Three detector families achieved high average precision on the processed split but showed markedly different raw-stream sensitivity and off-target alarm burdens. The raw hazard collection also showed a shift in a representative inter-band statistic relative to the training campaigns. We therefore developed a physics-guided pipeline based on background-normalized SWIR contrast, temporal consistency, geometric constraints, and range reporting. Under a disclosed method-informed diagnostic, the pipeline produced target-coincident responses in 24 of 25 raw hazard sequences with approximately 0.1 off-target alarm frames per minute. This operating point was not matched by the evaluated learned detectors in this collection. The pipeline was applied without retraining the learned detectors or performing test-time adaptation. Processed-split performance alone did not characterize raw-stream selectivity. Naturally formed black ice remains to be evaluated, but within the controlled surrogate setting, physics-guided sensing provided a selective complement to learned detectors.
- 8 September 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessPhysics-Guided Multispectral SWIR-Range Perception under Raw-Sensor Domain Shift
Supervised detectors may perform well on curated multispectral data yet behave differently on raw sensor streams. We investigate this gap using a vehicle-mounted four-band short-wave infrared (SWIR) range sensor, an annotated, processed collection, and 35 separately acquired nighttime sequences that contain road objects and controlled wet- and ice-like road-surface surrogates. Three detector families achieved high average precision on the processed split but showed markedly different raw-stream sensitivity and off-target alarm burdens. The raw hazard collection also showed a shift in a representative inter-band statistic relative to the training campaigns. We therefore developed a physics-guided pipeline based on background-normalized SWIR contrast, temporal consistency, geometric constraints, and range reporting. Under a disclosed method-informed diagnostic, the pipeline produced target-coincident responses in 24 of 25 raw hazard sequences with approximately 0.1 off-target alarm frames per minute. This operating point was not matched by the evaluated learned detectors in this collection. The pipeline was applied without retraining the learned detectors or performing test-time adaptation. Processed-split performance alone did not characterize raw-stream selectivity. Naturally formed black ice remains to be evaluated, but within the controlled surrogate setting, physics-guided sensing provided a selective complement to learned detectors.
- Open Access
- 7 August 2026
- Published by Elsevier in Results in Engineering
- Vol. 32
RM-MLLM: Robust scene understanding in adverse weather with RGB-multispectral fusion
- 7 August 2026
- Published by Elsevier in Results in Engineering
- Vol. 32
Open AccessRM-MLLM: Robust scene understanding in adverse weather with RGB-multispectral fusion
- Open Access
- 7 May 2026
- Published by Elsevier in SSRN Electronic Journal
RM-MLLM: Robust Scene Understanding in Adverse Weather with RGB-Multispectral Fusion
Multimodal large language models (MLLMs) for autonomous driving often become less reliable in adverse weather, partly because training data are dominated by clear, well-lit scenes, and most models primarily rely on RGB imagery. We address these issues with two contributions. First, we introduce DENSE-QA, a large-scale benchmark for adverse-weather scene understanding comprising 12,997 scenes and over 77,000 instruction-response pairs, built on synchronized RGB and multispectral imagery. Second, we propose RM-MLLM, an MLLM architecture with modality-specific encoders and an adaptive fusion module, trained with a three-stage progressive strategy to better integrate complementary sensor cues. Across multiple LLM backbones, RM-MLLM improves QA performance under adverse conditions. Using only RGB input, the proposed architecture and training already outperform all evaluated baselines, including prompt-augmented models, by over 11 percentage points. Incorporating RGB-multispectral fusion provides an additional 3-point gain concentrated on weather-sensitive tasks. We also observe that fine-tuning a strong RGB-only vision-language model on the same training data yields performance comparable to its zero-shot evaluation, suggesting that DENSE-QA fine-tuning alone does not close the gap to RM-MLLM and that multispectral input provides complementary value. We release the benchmark and models to support future all-weather autonomous driving research https://yonsei-stl.github.io/RM-MLLM.
- 7 May 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessRM-MLLM: Robust Scene Understanding in Adverse Weather with RGB-Multispectral Fusion
Multimodal large language models (MLLMs) for autonomous driving often become less reliable in adverse weather, partly because training data are dominated by clear, well-lit scenes, and most models primarily rely on RGB imagery. We address these issues with two contributions. First, we introduce DENSE-QA, a large-scale benchmark for adverse-weather scene understanding comprising 12,997 scenes and over 77,000 instruction-response pairs, built on synchronized RGB and multispectral imagery. Second, we propose RM-MLLM, an MLLM architecture with modality-specific encoders and an adaptive fusion module, trained with a three-stage progressive strategy to better integrate complementary sensor cues. Across multiple LLM backbones, RM-MLLM improves QA performance under adverse conditions. Using only RGB input, the proposed architecture and training already outperform all evaluated baselines, including prompt-augmented models, by over 11 percentage points. Incorporating RGB-multispectral fusion provides an additional 3-point gain concentrated on weather-sensitive tasks. We also observe that fine-tuning a strong RGB-only vision-language model on the same training data yields performance comparable to its zero-shot evaluation, suggesting that DENSE-QA fine-tuning alone does not close the gap to RM-MLLM and that multispectral input provides complementary value. We release the benchmark and models to support future all-weather autonomous driving research https://yonsei-stl.github.io/RM-MLLM.
- 26 February 2026
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 266
SpectraDiff: Enhancing the fidelity of Infrared Image Translation with object-aware diffusion
- 26 February 2026
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 266
SpectraDiff: Enhancing the fidelity of Infrared Image Translation with object-aware diffusion
- Open Access
- 17 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 215882-215896
Multi-Modal Cross-Attention-Based Multi-Label Weather and Severity Level Classification
Ensuring traffic safety in autonomous vehicles requires accurate weather and severity recognition, especially when operating within a defined operational design domain (ODD). However, most existing studies focus on single-label weather recognition and largely overlook weather severity classification. This gap in severity annotation limits the adaptability of autonomous systems under varied weather conditions. Current datasets are typically single-labeled and lack severity annotations, which are crucial for autonomous driving systems to adjust appropriately based on weather intensity. To address this data scarcity, we created a comprehensive multi-label weather dataset with light, moderate, and heavy severity levels based on quantitative visibility thresholds aligned with operational design domains (ODDs). We constructed this dataset by prompting multiple vision-language models (VLMs) to apply these criteria, then validated annotation quality against a human-labeled gold standard. This approach overcomes the limitations of existing datasets. On this new dataset, we introduce the Multi-Weather and Severity Classifier (MWSC), a novel multi-modal framework that simultaneously handles weather recognition and severity classification. MWSC employs a combination of cross-attention mechanisms to extract global and local image features and integrates them with text captions generated from a standardized, closed-vocabulary template for weather and severity to create robust feature associations for classifier training. Our experiments establish a new benchmark for environment perception in autonomous driving, where MWSC significantly outperforms widely-used baseline models across key metrics including mAP, accuracy, and F1-score, advancing vision-based techniques for safer and more reliable autonomous navigation in complex weather scenarios. The full dataset and code are publicly available in our repository at https://github.com/Yonsei-STL/MWSC.git, including licenses of all sources, human-annotated gold set, image lists and splits, prompts, and trained weights for reproducibility.
- 17 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 215882-215896
Open AccessMulti-Modal Cross-Attention-Based Multi-Label Weather and Severity Level Classification
Ensuring traffic safety in autonomous vehicles requires accurate weather and severity recognition, especially when operating within a defined operational design domain (ODD). However, most existing studies focus on single-label weather recognition and largely overlook weather severity classification. This gap in severity annotation limits the adaptability of autonomous systems under varied weather conditions. Current datasets are typically single-labeled and lack severity annotations, which are crucial for autonomous driving systems to adjust appropriately based on weather intensity. To address this data scarcity, we created a comprehensive multi-label weather dataset with light, moderate, and heavy severity levels based on quantitative visibility thresholds aligned with operational design domains (ODDs). We constructed this dataset by prompting multiple vision-language models (VLMs) to apply these criteria, then validated annotation quality against a human-labeled gold standard. This approach overcomes the limitations of existing datasets. On this new dataset, we introduce the Multi-Weather and Severity Classifier (MWSC), a novel multi-modal framework that simultaneously handles weather recognition and severity classification. MWSC employs a combination of cross-attention mechanisms to extract global and local image features and integrates them with text captions generated from a standardized, closed-vocabulary template for weather and severity to create robust feature associations for classifier training. Our experiments establish a new benchmark for environment perception in autonomous driving, where MWSC significantly outperforms widely-used baseline models across key metrics including mAP, accuracy, and F1-score, advancing vision-based techniques for safer and more reliable autonomous navigation in complex weather scenarios. The full dataset and code are publicly available in our repository at https://github.com/Yonsei-STL/MWSC.git, including licenses of all sources, human-annotated gold set, image lists and splits, prompts, and trained weights for reproducibility.
- 29 October 2025
- Published by Elsevier in Information Fusion
- Vol. 128
RASMD: RGB and SWIR multispectral driving dataset for robust perception in adverse conditions
- 29 October 2025
- Published by Elsevier in Information Fusion
- Vol. 128
RASMD: RGB and SWIR multispectral driving dataset for robust perception in adverse conditions