Hwanjun Song
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Hwanjun Song has a total of 79 co-authors. Here are the top co-authors by number of shared publications:
- Jae-Gil Lee
KAIST, Daejeon, Republic of Korea
10 shared publications Taewon Yun
Korea Advanced Institute of Science and Technology
9 shared publicationsJason Cai
AWS AI Labs
6 shared publicationsNicole Hee-Yeon Kim
Korea Advanced Institute of Science and Technology
6 shared publicationsHang Su
AWS AI Labs
5 shared publicationsMinjeong Ban
Korea Advanced Institute of Science and Technology
5 shared publications
Latest Publications
- Open Access
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1454-1465
No Logs, Only Catalogs: Catalog-to-Supervision Training for Recommendation with LLM-Simulated Preferences
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1454-1465
Open AccessNo Logs, Only Catalogs: Catalog-to-Supervision Training for Recommendation with LLM-Simulated Preferences
- Open Access
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1466-1477
TAG-Rec: Grounding LLM-based Recommendation via Trigger-Anchored GRPO
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1466-1477
Open AccessTAG-Rec: Grounding LLM-based Recommendation via Trigger-Anchored GRPO
- 10 September 2026
- Published by Springer Nature
- p. 375-393
CURE: Cumulative Knowledge Reuse for Efficient Device-Server Hybrid Inference in Vision-Language Models
- 10 September 2026
- Published by Springer Nature
- p. 375-393
CURE: Cumulative Knowledge Reuse for Efficient Device-Server Hybrid Inference in Vision-Language Models
- Open Access
- 21 February 2026
- Published by Association for Computing Machinery (ACM)
- p. 596-606
Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- 21 February 2026
- Published by Association for Computing Machinery (ACM)
- p. 596-606
Open AccessAligning Extraction and Generation for Robust Retrieval-Augmented Generation
- Open Access
Completing Missing Annotation: Multi-Agent Debate for Accurate and Scalable Relevant Assessment for IR Benchmarks
Information retrieval (IR) evaluation remains challenging due to incomplete IR benchmark datasets that contain unlabeled relevant chunks. While LLMs and LLM-human hybrid strategies reduce costly human effort, they remain prone to LLM overconfidence and ineffective AI-to-human escalation. To address this, we propose DREAM, a multi-round debate-based relevance assessment framework with LLM agents, built on opposing initial stances and iterative reciprocal critique. Through our agreement-based debate, it yields more accurate labeling for certain cases and more reliable AI-to-human escalation for uncertain ones, achieving 95.2% labeling accuracy with only 3.5% human involvement. Using DREAM, we build BRIDGE, a refined benchmark that mitigates evaluation bias and enables fairer retriever comparison by uncovering 29,824 missing relevant chunks. We then re-benchmark IR systems and extend evaluation to RAG, showing that unaddressed holes not only distort retriever rankings but also drive retrieval-generation misalignment. The relevance assessment framework is available at https: //github.com/DISL-Lab/DREAM-ICLR-26; and the BRIDGE dataset is available at https://github.com/DISL-Lab/BRIDGE-Benchmark.
Open AccessCompleting Missing Annotation: Multi-Agent Debate for Accurate and Scalable Relevant Assessment for IR Benchmarks
Information retrieval (IR) evaluation remains challenging due to incomplete IR benchmark datasets that contain unlabeled relevant chunks. While LLMs and LLM-human hybrid strategies reduce costly human effort, they remain prone to LLM overconfidence and ineffective AI-to-human escalation. To address this, we propose DREAM, a multi-round debate-based relevance assessment framework with LLM agents, built on opposing initial stances and iterative reciprocal critique. Through our agreement-based debate, it yields more accurate labeling for certain cases and more reliable AI-to-human escalation for uncertain ones, achieving 95.2% labeling accuracy with only 3.5% human involvement. Using DREAM, we build BRIDGE, a refined benchmark that mitigates evaluation bias and enables fairer retriever comparison by uncovering 29,824 missing relevant chunks. We then re-benchmark IR systems and extend evaluation to RAG, showing that unaddressed holes not only distort retriever rankings but also drive retrieval-generation misalignment. The relevance assessment framework is available at https: //github.com/DISL-Lab/DREAM-ICLR-26; and the BRIDGE dataset is available at https://github.com/DISL-Lab/BRIDGE-Benchmark.
- Open Access
Towards a Holistic and Automated Evaluation Framework for Multi-Level Comprehension of LLMs in Book-Length Contexts
We introduce HAMLET, a holistic and automated framework for evaluating the long-context comprehension of large language models (LLMs). HAMLET structures source texts into a three-level key-fact hierarchy at root-, branch-, and leaf-levels, and employs query-focused summarization to evaluate how well models recall and faithfully represent information at each level. To validate the reliability of our fully automated pipeline, we conduct a systematic human study, showing that our automatic evaluation achieves over 90% agreement with expert human judgments, while reducing the cost by up to 25 times. HAMLET reveals that LLMs struggle with fine-grained comprehension, especially at the leaf level, and are sensitive to positional effects like the lost-in-the-middle. Analytical queries pose greater challenges than narrative ones, and consistent performance gaps emerge between open-source and proprietary models, as well as across model scales. Our code and dataset are publicly available at https://github.com/DISL-Lab/HAMLET.
Open AccessTowards a Holistic and Automated Evaluation Framework for Multi-Level Comprehension of LLMs in Book-Length Contexts
We introduce HAMLET, a holistic and automated framework for evaluating the long-context comprehension of large language models (LLMs). HAMLET structures source texts into a three-level key-fact hierarchy at root-, branch-, and leaf-levels, and employs query-focused summarization to evaluate how well models recall and faithfully represent information at each level. To validate the reliability of our fully automated pipeline, we conduct a systematic human study, showing that our automatic evaluation achieves over 90% agreement with expert human judgments, while reducing the cost by up to 25 times. HAMLET reveals that LLMs struggle with fine-grained comprehension, especially at the leaf level, and are sensitive to positional effects like the lost-in-the-middle. Analytical queries pose greater challenges than narrative ones, and consistent performance gaps emerge between open-source and proprietary models, as well as across model scales. Our code and dataset are publicly available at https://github.com/DISL-Lab/HAMLET.
- Open Access
- 11 August 2025
- Published by Springer Nature in Data Mining and Knowledge Discovery
- Vol. 39 (5) , 1-29
Fine-grained multi-prompt essay scoring with multi-level disentanglement
The application of language models in essay scoring has gained significant attention in recent years, typically on the basis of evaluating a single model across multiple prompts. However, in a multi-prompt setup, it is crucial to understand the varying aspects of different prompts. In such settings, there exist notable variations even in a trait with the same name across prompts. This semantic variation on the same traits underscores the need to treat them differently at a fine-grained level according to each prompt. In this study, we propose a multi-level disentanglement framework for multi-prompt essay scoring, designed to achieve fine-grained disentanglement of semantic differences across such traits. Our method not only improves the quality of the essay scoring, but also reduces memory usage and latency. Experimental results highlight that our framework surpasses seven state-of-the-art essay scoring methods and large language model(LLM)-based zero-shot and few-shot approaches, achieving the highest agreement with human essay ratings.
- 11 August 2025
- Published by Springer Nature in Data Mining and Knowledge Discovery
- Vol. 39 (5) , 1-29
Open AccessFine-grained multi-prompt essay scoring with multi-level disentanglement
The application of language models in essay scoring has gained significant attention in recent years, typically on the basis of evaluating a single model across multiple prompts. However, in a multi-prompt setup, it is crucial to understand the varying aspects of different prompts. In such settings, there exist notable variations even in a trait with the same name across prompts. This semantic variation on the same traits underscores the need to treat them differently at a fine-grained level according to each prompt. In this study, we propose a multi-level disentanglement framework for multi-prompt essay scoring, designed to achieve fine-grained disentanglement of semantic differences across such traits. Our method not only improves the quality of the essay scoring, but also reduces memory usage and latency. Experimental results highlight that our framework surpasses seven state-of-the-art essay scoring methods and large language model(LLM)-based zero-shot and few-shot approaches, achieving the highest agreement with human essay ratings.
- Open Access
- 3 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 2162-2173
Bi-Modal Learning for Networked Time Series
- 3 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 2162-2173
Open AccessBi-Modal Learning for Networked Time Series
- Open Access
- 7 June 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the International AAAI Conference on Web and Social Media
- Vol. 19, 2539-2549
Mobility Networked Time-Series Forecasting Benchmark Datasets
Human mobility is crucial for urban planning (e.g., public transportation) and epidemic response strategies. However, existing research often neglects integrating comprehensive perspectives on spatial dynamics, temporal trends, and other contextual views due to the limitations of existing mobility datasets. To bridge this gap, we introduce MOBINS (MOBIlity Networked time Series), a novel dataset collection designed for networked time-series forecasting of dynamic human movements. MOBINS features diverse and explainable datasets that capture various mobility patterns across different transportation modes in four cities and two countries and cover both transportation and epidemic domains at the administrative area level. Our experiments with nine baseline methods reveal the significant impact of different model backbones on the proposed six datasets. We provide a valuable resource for advancing urban mobility research.
- 7 June 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the International AAAI Conference on Web and Social Media
- Vol. 19, 2539-2549
Open AccessMobility Networked Time-Series Forecasting Benchmark Datasets
Human mobility is crucial for urban planning (e.g., public transportation) and epidemic response strategies. However, existing research often neglects integrating comprehensive perspectives on spatial dynamics, temporal trends, and other contextual views due to the limitations of existing mobility datasets. To bridge this gap, we introduce MOBINS (MOBIlity Networked time Series), a novel dataset collection designed for networked time-series forecasting of dynamic human movements. MOBINS features diverse and explainable datasets that capture various mobility patterns across different transportation modes in four cities and two countries and cover both transportation and epidemic domains at the administrative area level. Our experiments with nine baseline methods reveal the significant impact of different model backbones on the proposed six datasets. We provide a valuable resource for advancing urban mobility research.
- Open Access
Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages
Evaluation frameworks for text summarization have evolved in terms of both domain coverage and metrics. However, existing benchmarks still lack domain-specific assessment criteria, remain predominantly English-centric, and face challenges with human annotation due to the complexity of reasoning. To address these, we introduce MSumBench, which provides a multi-dimensional, multi-domain evaluation of summarization in English and Chinese. It also incorporates specialized assessment criteria for each domain and leverages a multi-agent debate system to enhance annotation quality. By evaluating eight modern summarization models, we discover distinct performance patterns across domains and languages. We further examine large language models as summary evaluators, analyzing the correlation between their evaluation and summarization capabilities, and uncovering systematic bias in their assessment of self-generated summaries. Our benchmark dataset is publicly available at https://github.com/DISL-Lab/MSumBench.
Open AccessTowards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages
Evaluation frameworks for text summarization have evolved in terms of both domain coverage and metrics. However, existing benchmarks still lack domain-specific assessment criteria, remain predominantly English-centric, and face challenges with human annotation due to the complexity of reasoning. To address these, we introduce MSumBench, which provides a multi-dimensional, multi-domain evaluation of summarization in English and Chinese. It also incorporates specialized assessment criteria for each domain and leverages a multi-agent debate system to enhance annotation quality. By evaluating eight modern summarization models, we discover distinct performance patterns across domains and languages. We further examine large language models as summary evaluators, analyzing the correlation between their evaluation and summarization capabilities, and uncovering systematic bias in their assessment of self-generated summaries. Our benchmark dataset is publicly available at https://github.com/DISL-Lab/MSumBench.