Jiajun Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jiajun Liu has a total of 128 co-authors. Here are the top co-authors by number of shared publications:
- Sen Wang
The University of Queensland, Brisbane, Australia
13 shared publications - Branislav Kusy
DATA61, CSIRO
8 shared publications - Kun Zhao
Suncorp Grp, Brisbane, Qld 4000, Australia
5 shared publications - Raja Jurdak
Queensland University of Technology
5 shared publications - Ruihong Qiu
School of Electrical Engineering and Computer Science, The University of Queensland, St. Lucia, Brisbane, 4072, QLD, Australia
4 shared publications - Qi Wu
University of Adelaide, Adelaide, SA, Australia
3 shared publications
Latest Publications
- 24 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (8) , 9078-9092
Understanding the Effects of Projectors in Knowledge Distillation
Conventionally, during the knowledge distillation process (e.g., feature distillation), an additional projector is often required to perform feature transformation due to the dimension mismatch between the teacher and the student networks. Interestingly, we discovered that even if the student and the teacher have the same feature dimensions, adding a projector still helps to improve the distillation performance. In addition, projectors even improve logit distillation if we add them to the architecture too. Inspired by these surprising findings and the general lack of understanding of the projectors in the knowledge distillation process from existing literature, this paper investigates the implicit role that projectors play, but so far been overlooked. Our empirical study shows that the student with a projector (1) obtains a better trade-off between the training accuracy and the testing accuracy compared to the student without a projector when it has the same feature dimensions as the teacher, (2) better preserves its similarity to the teacher beyond shallow and numeric resemblance, from the view of Centered Kernel Alignment (CKA) [1], and (3) avoids being over-confident [2] as the teacher does at the testing phase. Motivated by the positive effects of projectors, we propose a projector ensemble-based feature distillation method to further improve distillation performance. Despite the simplicity of the proposed strategy, empirical results from the evaluation of classification tasks on benchmark datasets demonstrate the superior classification performance of our method on a broad range of teacher-student pairs and verify, from the aspects of CKA and model calibration that the student's features are of improved quality with the projector ensemble design. Code is available at https://github.com/chenyd7/PEFD.
- 24 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (8) , 9078-9092
Understanding the Effects of Projectors in Knowledge Distillation
Conventionally, during the knowledge distillation process (e.g., feature distillation), an additional projector is often required to perform feature transformation due to the dimension mismatch between the teacher and the student networks. Interestingly, we discovered that even if the student and the teacher have the same feature dimensions, adding a projector still helps to improve the distillation performance. In addition, projectors even improve logit distillation if we add them to the architecture too. Inspired by these surprising findings and the general lack of understanding of the projectors in the knowledge distillation process from existing literature, this paper investigates the implicit role that projectors play, but so far been overlooked. Our empirical study shows that the student with a projector (1) obtains a better trade-off between the training accuracy and the testing accuracy compared to the student without a projector when it has the same feature dimensions as the teacher, (2) better preserves its similarity to the teacher beyond shallow and numeric resemblance, from the view of Centered Kernel Alignment (CKA) [1], and (3) avoids being over-confident [2] as the teacher does at the testing phase. Motivated by the positive effects of projectors, we propose a projector ensemble-based feature distillation method to further improve distillation performance. Despite the simplicity of the proposed strategy, empirical results from the evaluation of classification tasks on benchmark datasets demonstrate the superior classification performance of our method on a broad range of teacher-student pairs and verify, from the aspects of CKA and model calibration that the student's features are of improved quality with the projector ensemble design. Code is available at https://github.com/chenyd7/PEFD.
- 12 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP (15209210) , 1-11
Video-Seer: A Hierarchical Scene Selection and Multi-View Analysis Approach for Zero-Shot Video Question Answering
Video Question Answering (VideoQA) is the task of answering a question in natural language by analyzing and understanding a video sequence. VideoQA is a typical manifestation of human intelligence, but it is one of the most complex activities for an artificial intelligence system. While vision-language large models excel in multi-modal understanding, they often struggle to capture causal and temporal dependencies in VideoQA. In this paper, we introduce Video-Seer, a novel training-free model inspired by human cognition for zero-shot video question answering. Video-Seer employs hierarchical scene selection through dynamic segmentation and multi-modal relevance scoring to identify question-relevant scenes. It dynamically allocates keyframes based on scene importance, prioritizing high-relevance regions, and integrates fine-grained keyframe grids with global sparse sampling for comprehensive multi-view modeling. Evaluated on challenging multiple-choice (NExT-QA, IntentQA, EgoSchema) and open-ended (MSVD-QA, MSRVTT-QA) VideoQA datasets, Video-Seer achieves accuracies of 71.4%, 65.5%, and 58.6% on multiple-choice tasks, surpassing comparable trained and zero-shot models using a lightweight 7B-parameter language model. These results demonstrate Video-Seer's robust video understanding, reasoning capabilities, and strong generalization without requiring extensive video-question pair training.
- 12 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP (15209210) , 1-11
Video-Seer: A Hierarchical Scene Selection and Multi-View Analysis Approach for Zero-Shot Video Question Answering
Video Question Answering (VideoQA) is the task of answering a question in natural language by analyzing and understanding a video sequence. VideoQA is a typical manifestation of human intelligence, but it is one of the most complex activities for an artificial intelligence system. While vision-language large models excel in multi-modal understanding, they often struggle to capture causal and temporal dependencies in VideoQA. In this paper, we introduce Video-Seer, a novel training-free model inspired by human cognition for zero-shot video question answering. Video-Seer employs hierarchical scene selection through dynamic segmentation and multi-modal relevance scoring to identify question-relevant scenes. It dynamically allocates keyframes based on scene importance, prioritizing high-relevance regions, and integrates fine-grained keyframe grids with global sparse sampling for comprehensive multi-view modeling. Evaluated on challenging multiple-choice (NExT-QA, IntentQA, EgoSchema) and open-ended (MSVD-QA, MSRVTT-QA) VideoQA datasets, Video-Seer achieves accuracies of 71.4%, 65.5%, and 58.6% on multiple-choice tasks, surpassing comparable trained and zero-shot models using a lightweight 7B-parameter language model. These results demonstrate Video-Seer's robust video understanding, reasoning capabilities, and strong generalization without requiring extensive video-question pair training.
- 23 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 13 (11) , 23025-23042
DFedDG2: Distribution-Guided Gossip-Based Generalizable and Communication-Efficient Decentralized Federated Learning
Traditional federated learning (FL) focuses on collaborative global model training while ensuring privacy and personalization. Decentralized FL (DFL), a variant of FL, allows clients to independently manage and optimize local models without a central server. DFL reduces centralized communication bottlenecks and vulnerability to server failures or attacks. However, because the optimization dynamics change and there is no global model, generalization can suffer, making effective learning under data and model heterogeneity a critical challenge in DFL. Despite growing interest in DFL, the lack of distributional and uncertainty modeling in the literature limits reliability and effective generalization in nonidentically distributed (IID) settings. In this work, we propose DFedDG2, a personalized FL framework that operates within a peer-to-peer protocol. DFedDG2 offers the technical advantage of modeling each client’s local data distribution and exchanging this information with one-hop neighbors. In the decentralized network, clients perform distribution-aware gossip, where statistically similar clients exert greater influence to drive global alignment. This likelihood-weighted mixing fuses only a handful of vectors and scalars, significantly reducing communication costs while aligning semantic spaces across the network and enabling personalized training at the edge. In addition, theoretically, we prove that DFedDG2 achieves a sublinear convergence rate while the consensus error decays at a geometric rate under well-principled properties of gossip. Unlike the label-only non-IID experiments in DFL literature, we conduct extensive experiments on multiple data non-IID scenarios, topology variations, model heterogeneity, and uncertainty quantification, demonstrating the practical advantages of DFedDG2. Our results show that DFedDG2 not only achieves communication efficiency but also provides better generalization and improved reliability compared to state-of-the-art approaches.
- 23 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 13 (11) , 23025-23042
DFedDG2: Distribution-Guided Gossip-Based Generalizable and Communication-Efficient Decentralized Federated Learning
Traditional federated learning (FL) focuses on collaborative global model training while ensuring privacy and personalization. Decentralized FL (DFL), a variant of FL, allows clients to independently manage and optimize local models without a central server. DFL reduces centralized communication bottlenecks and vulnerability to server failures or attacks. However, because the optimization dynamics change and there is no global model, generalization can suffer, making effective learning under data and model heterogeneity a critical challenge in DFL. Despite growing interest in DFL, the lack of distributional and uncertainty modeling in the literature limits reliability and effective generalization in nonidentically distributed (IID) settings. In this work, we propose DFedDG2, a personalized FL framework that operates within a peer-to-peer protocol. DFedDG2 offers the technical advantage of modeling each client’s local data distribution and exchanging this information with one-hop neighbors. In the decentralized network, clients perform distribution-aware gossip, where statistically similar clients exert greater influence to drive global alignment. This likelihood-weighted mixing fuses only a handful of vectors and scalars, significantly reducing communication costs while aligning semantic spaces across the network and enabling personalized training at the edge. In addition, theoretically, we prove that DFedDG2 achieves a sublinear convergence rate while the consensus error decays at a geometric rate under well-principled properties of gossip. Unlike the label-only non-IID experiments in DFL literature, we conduct extensive experiments on multiple data non-IID scenarios, topology variations, model heterogeneity, and uncertainty quantification, demonstrating the practical advantages of DFedDG2. Our results show that DFedDG2 not only achieves communication efficiency but also provides better generalization and improved reliability compared to state-of-the-art approaches.
- Open Access
- 6 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 240-250
MARCO: A Cooperative Knowledge Transfer Framework for Personalized Cross-domain Recommendations
- 6 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 240-250
Open AccessMARCO: A Cooperative Knowledge Transfer Framework for Personalized Cross-domain Recommendations
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 12492-12500
Queryable 3D Scene Representation: A Multi-Modal Framework for Semantic Reasoning and Robotic Task Planning
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 12492-12500
Open AccessQueryable 3D Scene Representation: A Multi-Modal Framework for Semantic Reasoning and Robotic Task Planning
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 277-285
Querying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNet
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 277-285
Open AccessQuerying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNet
- Open Access
Beyond Static LLM Policies: Imitation-Enhanced Reinforcement Learning for Recommendation
Recommender systems (RecSys) have become critical tools for enhancing user engagement by delivering personalized content across diverse digital platforms. Recent advancements in large language models (LLMs) demonstrate significant potential for improving RecSys, primarily due to their exceptional generalization capabilities and sophisticated contextual understanding, which facilitate the generation of flexible and interpretable recommendations. However, the direct deployment of LLMs as primary recommendation policies presents notable challenges, including persistent latency issues stemming from frequent API calls and inherent model limitations such as hallucinations and biases. To address these issues, this paper proposes a novel offline reinforcement learning (RL) framework that leverages imitation learning from LLM-generated trajectories. Specifically, inverse reinforcement learning is employed to extract robust reward models from LLM demonstrations. This approach negates the need for LLM fine-tuning, thereby substantially reducing computational overhead. Simultaneously, the RL policy is guided by the cumulative rewards derived from these demonstrations, effectively transferring the semantic insights captured by the LLM. Comprehensive experiments conducted on two benchmark datasets validate the effectiveness of the proposed method, demonstrating superior performance when compared against state-of-the-art RL-based and in-context learning baselines. The code can be found at https://github.com/ArronDZhang/IL-Rec.
Open AccessBeyond Static LLM Policies: Imitation-Enhanced Reinforcement Learning for Recommendation
Recommender systems (RecSys) have become critical tools for enhancing user engagement by delivering personalized content across diverse digital platforms. Recent advancements in large language models (LLMs) demonstrate significant potential for improving RecSys, primarily due to their exceptional generalization capabilities and sophisticated contextual understanding, which facilitate the generation of flexible and interpretable recommendations. However, the direct deployment of LLMs as primary recommendation policies presents notable challenges, including persistent latency issues stemming from frequent API calls and inherent model limitations such as hallucinations and biases. To address these issues, this paper proposes a novel offline reinforcement learning (RL) framework that leverages imitation learning from LLM-generated trajectories. Specifically, inverse reinforcement learning is employed to extract robust reward models from LLM demonstrations. This approach negates the need for LLM fine-tuning, thereby substantially reducing computational overhead. Simultaneously, the RL policy is guided by the cumulative rewards derived from these demonstrations, effectively transferring the semantic insights captured by the LLM. Comprehensive experiments conducted on two benchmark datasets validate the effectiveness of the proposed method, demonstrating superior performance when compared against state-of-the-art RL-based and in-context learning baselines. The code can be found at https://github.com/ArronDZhang/IL-Rec.
- Open Access
On-the-Fly Object-aware Representative Point Selection in Point Cloud
Point clouds are essential for object modeling and play a critical role in assisting driving tasks for autonomous vehicles (AVs). However, the significant volume of data generated by AVs creates challenges for storage, bandwidth, and processing cost. To tackle these challenges, we propose a representative point selection framework for point cloud downsampling, which preserves critical object-related information while effectively filtering out irrelevant background points. Our method involves two steps: (1) Object Presence Detection, where we introduce an unsupervised density peak-based classifier and a supervised Naïve Bayes classifier to handle diverse scenarios, and (2) Sampling Budget Allocation, where we propose a strategy that selects object-relevant points while maintaining a high retention rate of object information. Extensive experiments on the KITTI and nuScenes datasets demonstrate that our method consistently outperforms state-of-the-art baselines in both efficiency and effectiveness across varying sampling rates. As a model-agnostic solution, our approach integrates seamlessly with diverse downstream models, making it a valuable and scalable addition to the 3D point cloud downsampling toolkit for AV applications.
Open AccessOn-the-Fly Object-aware Representative Point Selection in Point Cloud
Point clouds are essential for object modeling and play a critical role in assisting driving tasks for autonomous vehicles (AVs). However, the significant volume of data generated by AVs creates challenges for storage, bandwidth, and processing cost. To tackle these challenges, we propose a representative point selection framework for point cloud downsampling, which preserves critical object-related information while effectively filtering out irrelevant background points. Our method involves two steps: (1) Object Presence Detection, where we introduce an unsupervised density peak-based classifier and a supervised Naïve Bayes classifier to handle diverse scenarios, and (2) Sampling Budget Allocation, where we propose a strategy that selects object-relevant points while maintaining a high retention rate of object information. Extensive experiments on the KITTI and nuScenes datasets demonstrate that our method consistently outperforms state-of-the-art baselines in both efficiency and effectiveness across varying sampling rates. As a model-agnostic solution, our approach integrates seamlessly with diverse downstream models, making it a valuable and scalable addition to the 3D point cloud downsampling toolkit for AV applications.
- Open Access
Quantifying and Narrowing the Unknown: Interactive Text-to-Video Retrieval via Uncertainty Minimization
Despite recent advances, Text-to-video retrieval (TVR) is still hindered by multiple inherent uncertainties, such as ambiguous textual queries, indistinct text-video mappings, and low-quality video frames. Although interactive systems have emerged to address these challenges by refining user intent through clarifying questions, current methods typically rely on heuristic or ad-hoc strategies without explicitly quantifying these uncertainties, limiting their effectiveness. Motivated by this gap, we propose UMIVR, an Uncertainty-Minimizing Interactive Text-to-Video Retrieval framework that explicitly quantifies three critical uncertainties-text ambiguity, mapping uncertainty, and frame uncertainty-via principled, training-free metrics: semantic entropy-based Text Ambiguity Score (TAS), Jensen-Shannon divergence-based Mapping Uncertainty Score (MUS), and a Temporal Quality-based Frame Sampler (TQFS). By adaptively generating targeted clarifying questions guided by these uncertainty measures, UMIVR iteratively refines user queries, significantly reducing retrieval ambiguity. Extensive experiments on multiple benchmarks validate UMIVR's effectiveness, achieving notable gains in Recall@1 (69.2\% after 10 interactive rounds) on the MSR-VTT-1k dataset, thereby establishing an uncertainty-minimizing foundation for interactive TVR.
Open AccessQuantifying and Narrowing the Unknown: Interactive Text-to-Video Retrieval via Uncertainty Minimization
Despite recent advances, Text-to-video retrieval (TVR) is still hindered by multiple inherent uncertainties, such as ambiguous textual queries, indistinct text-video mappings, and low-quality video frames. Although interactive systems have emerged to address these challenges by refining user intent through clarifying questions, current methods typically rely on heuristic or ad-hoc strategies without explicitly quantifying these uncertainties, limiting their effectiveness. Motivated by this gap, we propose UMIVR, an Uncertainty-Minimizing Interactive Text-to-Video Retrieval framework that explicitly quantifies three critical uncertainties-text ambiguity, mapping uncertainty, and frame uncertainty-via principled, training-free metrics: semantic entropy-based Text Ambiguity Score (TAS), Jensen-Shannon divergence-based Mapping Uncertainty Score (MUS), and a Temporal Quality-based Frame Sampler (TQFS). By adaptively generating targeted clarifying questions guided by these uncertainty measures, UMIVR iteratively refines user queries, significantly reducing retrieval ambiguity. Extensive experiments on multiple benchmarks validate UMIVR's effectiveness, achieving notable gains in Recall@1 (69.2\% after 10 interactive rounds) on the MSR-VTT-1k dataset, thereby establishing an uncertainty-minimizing foundation for interactive TVR.
- Open Access
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 2192-2202
DARLR: Dual-Agent Offline Reinforcement Learning for Recommender Systems with Dynamic Reward
- 13 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 2192-2202
Open AccessDARLR: Dual-Agent Offline Reinforcement Learning for Recommender Systems with Dynamic Reward