Junbin Xiao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Junbin Xiao has a total of 59 co-authors. Here are the top co-authors by number of shared publications:
- Tat-Seng Chua
National University of Singapore, Singapore, Singapore
16 shared publications - Angela Yao
National University of Singapore, Singapore, Singapore
11 shared publications - Yicong Li
National University of Singapore, Singapore, Singapore
8 shared publications - Wei Ji
Nanjing University, Suzhou, China
5 shared publications - Pan Zhou
Singapore Management University
3 shared publications - Shuicheng Yan
National University of Singapore
3 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 17925-17926
Video Understanding and Beyond: Reasoning, Streaming, and Spatial Intelligence
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 17925-17926
Open AccessVideo Understanding and Beyond: Reasoning, Streaming, and Spatial Intelligence
- 8 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 1417-1430
Scene-Text Grounding for Text-Based Video Question Answering
Existing efforts in text-based video question answering (TextVideoQA) are criticized for their opaque decision-making and heavy reliance on scene-text recognition. In this paper, we propose to study Grounded TextVideoQA by forcing models to answer questions and spatio-temporally localize the relevant scene-text regions, thus decoupling QA from scene-text recognition and promoting research towards interpretable QA. The task has three-fold significance. First, it encourages scene-text evidence versus other short-cuts for answer predictions. Second, it directly accepts scene-text regions as visual answers, thus circumventing the problem of ineffective answer evaluation by stringent string matching. Third, it isolates the challenges inherited in VideoQA and scene-text recognition. This enables the diagnosis of the root causes for failure predictions, e.g., wrong QA or wrong scene-text recognition? To achieve Grounded TextVideoQA, we propose the T2S-QA model that highlights a disentangled temporal-to-spatial contrastive learning strategy for weakly-supervised scene-text grounding and grounded TextVideoQA. To facilitate evaluation, we construct a new dataset ViTXT-GQA which features 52K scene-text bounding boxes within 2.2K temporal segments related to 2K questions and 729 videos. With ViTXT-GQA, we perform extensive experiments and demonstrate the severe limitations of existing techniques in Grounded TextVideoQA. While T2S-QA achieves superior results, the large performance gap with human leaves ample space for improvement. Our further analysis of oracle scene-text inputs posits that the major challenge is scene-text recognition. To advance the research of Grounded TextVideoQA, our dataset and code are at https://github.com/zhousheng97/ViTXT-GQA.git
- 8 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 1417-1430
Scene-Text Grounding for Text-Based Video Question Answering
Existing efforts in text-based video question answering (TextVideoQA) are criticized for their opaque decision-making and heavy reliance on scene-text recognition. In this paper, we propose to study Grounded TextVideoQA by forcing models to answer questions and spatio-temporally localize the relevant scene-text regions, thus decoupling QA from scene-text recognition and promoting research towards interpretable QA. The task has three-fold significance. First, it encourages scene-text evidence versus other short-cuts for answer predictions. Second, it directly accepts scene-text regions as visual answers, thus circumventing the problem of ineffective answer evaluation by stringent string matching. Third, it isolates the challenges inherited in VideoQA and scene-text recognition. This enables the diagnosis of the root causes for failure predictions, e.g., wrong QA or wrong scene-text recognition? To achieve Grounded TextVideoQA, we propose the T2S-QA model that highlights a disentangled temporal-to-spatial contrastive learning strategy for weakly-supervised scene-text grounding and grounded TextVideoQA. To facilitate evaluation, we construct a new dataset ViTXT-GQA which features 52K scene-text bounding boxes within 2.2K temporal segments related to 2K questions and 729 videos. With ViTXT-GQA, we perform extensive experiments and demonstrate the severe limitations of existing techniques in Grounded TextVideoQA. While T2S-QA achieves superior results, the large performance gap with human leaves ample space for improvement. Our further analysis of oracle scene-text inputs posits that the major challenge is scene-text recognition. To advance the research of Grounded TextVideoQA, our dataset and code are at https://github.com/zhousheng97/ViTXT-GQA.git
- Open Access
- 6 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Can Multimodal Large Language Models Understand Human Values in Videos?
- 6 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Open AccessCan Multimodal Large Language Models Understand Human Values in Videos?
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 14346-14347
Video Question Answering and Beyond
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 14346-14347
Open AccessVideo Question Answering and Beyond
- 20 September 2025
- Published by Springer Nature
- p. 669-679
Unleashing the Power of LLMs for Medical Video Answer Localization
- 20 September 2025
- Published by Springer Nature
- p. 669-679
Unleashing the Power of LLMs for Medical Video Answer Localization
- Open Access
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 884-894
Question-Answering Dense Video Events
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 884-894
Open AccessQuestion-Answering Dense Video Events
- Open Access
- 30 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 889-898
Bottom-Up and Top-Down Thoughts for Visual Intention Grounding
- 30 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 889-898
Open AccessBottom-Up and Top-Down Thoughts for Visual Intention Grounding
- Open Access
Visual Intention Grounding for Egocentric Assistants
Visual grounding associates textual descriptions with objects in an image. Conventional methods target third-person image inputs and named object queries. In applications such as AI assistants, the perspective shifts -- inputs are egocentric, and objects may be referred to implicitly through needs and intentions. To bridge this gap, we introduce EgoIntention, the first dataset for egocentric visual intention grounding. EgoIntention challenges multimodal LLMs to 1) understand and ignore unintended contextual objects and 2) reason about uncommon object functionalities. Benchmark results show that current models misidentify context objects and lack affordance understanding in egocentric views. We also propose Reason-to-Ground (RoG) instruction tuning; it enables hybrid training with normal descriptions and egocentric intentions with a chained intention reasoning and object grounding mechanism. RoG significantly outperforms naive finetuning and hybrid training on EgoIntention, while maintaining or slightly improving naive description grounding. This advancement enables unified visual grounding for egocentric and exocentric visual inputs while handling explicit object queries and implicit human intentions.
Open AccessVisual Intention Grounding for Egocentric Assistants
Visual grounding associates textual descriptions with objects in an image. Conventional methods target third-person image inputs and named object queries. In applications such as AI assistants, the perspective shifts -- inputs are egocentric, and objects may be referred to implicitly through needs and intentions. To bridge this gap, we introduce EgoIntention, the first dataset for egocentric visual intention grounding. EgoIntention challenges multimodal LLMs to 1) understand and ignore unintended contextual objects and 2) reason about uncommon object functionalities. Benchmark results show that current models misidentify context objects and lack affordance understanding in egocentric views. We also propose Reason-to-Ground (RoG) instruction tuning; it enables hybrid training with normal descriptions and egocentric intentions with a chained intention reasoning and object grounding mechanism. RoG significantly outperforms naive finetuning and hybrid training on EgoIntention, while maintaining or slightly improving naive description grounding. This advancement enables unified visual grounding for egocentric and exocentric visual inputs while handling explicit object queries and implicit human intentions.
- 18 February 2025
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 133 (7) , 3970-3993
VideoQA in the Era of LLMs: An Empirical Study
- 18 February 2025
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 133 (7) , 3970-3993
VideoQA in the Era of LLMs: An Empirical Study
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 4331-4340
Causal-driven Large Language Models with Faithful Reasoning for Knowledge Question Answering
In Large Language Models (LLMs), text generation that involves knowledge representation is often fraught with the risk of "hallucinations'', where models confidently produce erroneous or fabricated content. These inaccuracies often stem from intrinsic biases in the pre-training stage or from the incorporation of human preference biases during the fine-tuning process. To mitigate these issues, we take inspiration from Goldman's causal theory of knowledge, which asserts that knowledge is not merely about having a true belief but also involves a causal connection between the belief and the truth of the proposition. We instantiate this theory within the context of Knowledge Question Answering (KQA) by constructing a causal graph that delineates the pathways between the candidate knowledge and belief. Through the application of the do-calculus rules from structural causal models, we devise an unbiased estimation framework based on this causal graph, thereby establishing a methodology for knowledge modeling grounded in causal inference. The resulting CORE framework (short for "Causal knOwledge REasoning'') is comprised of four essential components: question answering, causal reasoning, belief scoring, and refinement. Together, they synergistically improve the KQA system by fostering faithful reasoning and introspection. Extensive experiments are conducted on ScienceQA and HotpotQA datasets, which demonstrate the effectiveness and rationality of the CORE framework.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 4331-4340
Open AccessCausal-driven Large Language Models with Faithful Reasoning for Knowledge Question Answering
In Large Language Models (LLMs), text generation that involves knowledge representation is often fraught with the risk of "hallucinations'', where models confidently produce erroneous or fabricated content. These inaccuracies often stem from intrinsic biases in the pre-training stage or from the incorporation of human preference biases during the fine-tuning process. To mitigate these issues, we take inspiration from Goldman's causal theory of knowledge, which asserts that knowledge is not merely about having a true belief but also involves a causal connection between the belief and the truth of the proposition. We instantiate this theory within the context of Knowledge Question Answering (KQA) by constructing a causal graph that delineates the pathways between the candidate knowledge and belief. Through the application of the do-calculus rules from structural causal models, we devise an unbiased estimation framework based on this causal graph, thereby establishing a methodology for knowledge modeling grounded in causal inference. The resulting CORE framework (short for "Causal knOwledge REasoning'') is comprised of four essential components: question answering, causal reasoning, belief scoring, and refinement. Together, they synergistically improve the KQA system by fostering faithful reasoning and introspection. Extensive experiments are conducted on ScienceQA and HotpotQA datasets, which demonstrate the effectiveness and rationality of the CORE framework.