Zhihao Wu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Zhihao Wu has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
- Youfang Lin
School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China and Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing, China
6 shared publications Youfang Lin
Affiliation School of Computer and Information Technology, Beijing JiaoTong University, Beijing, China
5 shared publications- Kai Lv
Beijing Jiaotong University
4 shared publications - Shuo Wang
Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, School of Computer Science & Technology, Beijing Jiaotong University, China
4 shared publications - Xiaobo Hu
School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China
3 shared publications Jinwen Wang
School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China
2 shared publications
Latest Publications
- 11 July 2026
- Published by Elsevier in Neural Networks
- Vol. 205 (Pt A) , 109378
A Unified Basis Decomposition Framework for Addressing the Federated Learning Trilemma: Communication-efficiency, Personalization, and Privacy
- 11 July 2026
- Published by Elsevier in Neural Networks
- Vol. 205 (Pt A) , 109378
A Unified Basis Decomposition Framework for Addressing the Federated Learning Trilemma: Communication-efficiency, Personalization, and Privacy
- Open Access
- 1 August 2024
- Published by International Joint Conferences on Artificial Intelligence
- Vol. 2, 1389-1397
How to Learn Domain-Invariant Representations for Visual Reinforcement Learning: An Information-Theoretical Perspective
Despite the impressive success in visual control challenges, Visual Reinforcement Learning (VRL) policies have struggled to generalize to other scenarios. Existing works attempt to empirically improve the generalization capability, lacking theoretical support. In this work, we explore how to learn domain-invariant representations for VRL from an information-theoretical perspective. Specifically, we identify three Mutual Information (MI) terms. These terms highlight that a robust representation should preserve domain invariant information (return and dynamic transition) under significant observation perturbation. Furthermore, we relax the MI terms to derive three components for implementing a practical Mutual Information-based Invariant Representation (MIIR) algorithm for VRL. Extensive experiments demonstrate that MIIR achieves state-of-the-art generalization performance and the best sample efficiency in the DeepMind Control suite, Robotic Manipulation, and Carla.
- 1 August 2024
- Published by International Joint Conferences on Artificial Intelligence
- Vol. 2, 1389-1397
Open AccessHow to Learn Domain-Invariant Representations for Visual Reinforcement Learning: An Information-Theoretical Perspective
Despite the impressive success in visual control challenges, Visual Reinforcement Learning (VRL) policies have struggled to generalize to other scenarios. Existing works attempt to empirically improve the generalization capability, lacking theoretical support. In this work, we explore how to learn domain-invariant representations for VRL from an information-theoretical perspective. Specifically, we identify three Mutual Information (MI) terms. These terms highlight that a robust representation should preserve domain invariant information (return and dynamic transition) under significant observation perturbation. Furthermore, we relax the MI terms to derive three components for implementing a practical Mutual Information-based Invariant Representation (MIIR) algorithm for VRL. Extensive experiments demonstrate that MIIR achieves state-of-the-art generalization performance and the best sample efficiency in the DeepMind Control suite, Robotic Manipulation, and Carla.
- Open Access
- 16 May 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 20 (7) , 1-22
Building Category Graphs Representation with Spatial and Temporal Attention for Visual Navigation
Given an object of interest, visual navigation aims to reach the object’s location based on a sequence of partial observations. To this end, an agent needs to (1) acquire specific knowledge about the relations of object categories in the world during training and (2) locate the target object based on the pre-learned object category relations and its trajectory in the current unseen environment. In this article, we propose a Category Relation Graph (CRG) to learn the knowledge of object category layout relations and a Temporal-Spatial-Region attention (TSR) architecture to perceive the long-term spatial-temporal dependencies of objects, aiding navigation. We establish CRG to learn prior knowledge of object layout and deduce the positions of specific objects. Subsequently, we propose the TSR architecture to capture relationships among objects in temporal, spatial, and regions within observation trajectories. Specifically, we implement a Temporal attention module (T) to model the temporal structure of the observation sequence, implicitly encoding historical moving or trajectory information. Then, a Spatial attention module (S) uncovers the spatial context of the current observation objects based on CRG and past observations. Last, a Region attention module (R) shifts the attention to the target-relevant region. Leveraging the visual representation extracted by our method, the agent accurately perceives the environment and easily learns a superior navigation policy. Experiments on AI2-THOR demonstrate that our CRG-TSR method significantly outperforms existing methods in both effectiveness and efficiency. The supplementary material includes the code and will be publicly available.
- 16 May 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 20 (7) , 1-22
Open AccessBuilding Category Graphs Representation with Spatial and Temporal Attention for Visual Navigation
Given an object of interest, visual navigation aims to reach the object’s location based on a sequence of partial observations. To this end, an agent needs to (1) acquire specific knowledge about the relations of object categories in the world during training and (2) locate the target object based on the pre-learned object category relations and its trajectory in the current unseen environment. In this article, we propose a Category Relation Graph (CRG) to learn the knowledge of object category layout relations and a Temporal-Spatial-Region attention (TSR) architecture to perceive the long-term spatial-temporal dependencies of objects, aiding navigation. We establish CRG to learn prior knowledge of object layout and deduce the positions of specific objects. Subsequently, we propose the TSR architecture to capture relationships among objects in temporal, spatial, and regions within observation trajectories. Specifically, we implement a Temporal attention module (T) to model the temporal structure of the observation sequence, implicitly encoding historical moving or trajectory information. Then, a Spatial attention module (S) uncovers the spatial context of the current observation objects based on CRG and past observations. Last, a Region attention module (R) shifts the attention to the target-relevant region. Leveraging the visual representation extracted by our method, the agent accurately perceives the environment and easily learns a superior navigation policy. Experiments on AI2-THOR demonstrate that our CRG-TSR method significantly outperforms existing methods in both effectiveness and efficiency. The supplementary material includes the code and will be publicly available.
- Open Access
- 24 March 2024
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (6) , 5590-5598
What Effects the Generalization in Visual Reinforcement Learning: Policy Consistency with Truncated Return Prediction
In visual Reinforcement Learning (RL), the challenge of generalization to new environments is paramount. This study pioneers a theoretical analysis of visual RL generalization, establishing an upper bound on the generalization objective, encompassing policy divergence and Bellman error components. Motivated by this analysis, we propose maintaining the cross-domain consistency for each policy in the policy space, which can reduce the divergence of the learned policy during the test. In practice, we introduce the Truncated Return Prediction (TRP) task, promoting cross-domain policy consistency by predicting truncated returns of historical trajectories. Moreover, we also propose a Transformer-based predictor for this auxiliary task. Extensive experiments on DeepMind Control Suite and Robotic Manipulation tasks demonstrate that TRP achieves state-of-the-art generalization performance. We further demonstrate that TRP outperforms previous methods in terms of sample efficiency during training.
- 24 March 2024
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (6) , 5590-5598
Open AccessWhat Effects the Generalization in Visual Reinforcement Learning: Policy Consistency with Truncated Return Prediction
In visual Reinforcement Learning (RL), the challenge of generalization to new environments is paramount. This study pioneers a theoretical analysis of visual RL generalization, establishing an upper bound on the generalization objective, encompassing policy divergence and Bellman error components. Motivated by this analysis, we propose maintaining the cross-domain consistency for each policy in the policy space, which can reduce the divergence of the learned policy during the test. In practice, we introduce the Truncated Return Prediction (TRP) task, promoting cross-domain policy consistency by predicting truncated returns of historical trajectories. Moreover, we also propose a Transformer-based predictor for this auxiliary task. Extensive experiments on DeepMind Control Suite and Robotic Manipulation tasks demonstrate that TRP achieves state-of-the-art generalization performance. We further demonstrate that TRP outperforms previous methods in terms of sample efficiency during training.
- Open Access
Agent-Centric Relation Graph for Object Visual Navigation
Object visual navigation aims to steer an agent toward a target object based on visual observations. It is highly desirable to reasonably perceive the environment and accurately control the agent. In the navigation task, we introduce an Agent-Centric Relation Graph (ACRG) for learning the visual representation based on the relationships in the environment. ACRG is a highly effective structure that consists of two relationships, i.e., the horizontal relationship among objects and the distance relationship between the agent and objects . On the one hand, we design the Object Horizontal Relationship Graph (OHRG) that stores the relative horizontal location among objects. On the other hand, we propose the Agent-Target Distance Relationship Graph (ATDRG) that enables the agent to perceive the distance between the target and objects. For ATDRG, we utilize image depth to obtain the target distance and imply the vertical location to capture the distance relationship among objects in the vertical direction. With the above graphs, the agent can perceive the environment and output navigation actions. Experimental results in the artificial environment AI2-THOR demonstrate that ACRG significantly outperforms other state-of-the-art methods in unseen testing environments.
Open AccessAgent-Centric Relation Graph for Object Visual Navigation
Object visual navigation aims to steer an agent toward a target object based on visual observations. It is highly desirable to reasonably perceive the environment and accurately control the agent. In the navigation task, we introduce an Agent-Centric Relation Graph (ACRG) for learning the visual representation based on the relationships in the environment. ACRG is a highly effective structure that consists of two relationships, i.e., the horizontal relationship among objects and the distance relationship between the agent and objects . On the one hand, we design the Object Horizontal Relationship Graph (OHRG) that stores the relative horizontal location among objects. On the other hand, we propose the Agent-Target Distance Relationship Graph (ATDRG) that enables the agent to perceive the distance between the target and objects. For ATDRG, we utilize image depth to obtain the target distance and imply the vertical location to capture the distance relationship among objects in the vertical direction. With the above graphs, the agent can perceive the environment and output navigation actions. Experimental results in the artificial environment AI2-THOR demonstrate that ACRG significantly outperforms other state-of-the-art methods in unseen testing environments.
- Open Access
- 24 June 2022
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 17 (6) , e0269651
TRR: Target-relation regulated network for sequential recommendation
Item co-occurrence is an important pattern in recommendation. Due to the difference in correlation, the matching degrees between the target and historical items vary. The higher the matching degree, the greater probability they co-occur. Recently, the recommendation performance has been greatly improved by leveraging item relations. As an important bond imposed by relations, these connected items should have a strong correlation in the calculation of certain measures. This kind of correlation can be the biased knowledge that benefits parameter training. Specifically, we focus on tuples containing the target item and latest relational items that have relations such as complement or substitute with the target item in user’s behavior sequence. Such close relations mean the matching degrees between relational items and historical items should be highly affected by that of the target item and historical items. For example, given a relational item having relation complement with the target item, if the target item has high matching degrees with some items in user’s behavior sequence, this complementary item should behave similarly for the co-occurrence of complementary items. Under guidance of the above thought, in this work, we propose a target-relation regulated mechanism which converts the biased knowledge of high correlation of matching degrees into a regulation. It integrates the target item and relational items in history as a whole to characterize the matching score between the target item and historical items. Experiments conducted on three real-world datasets demonstrate that our model can significantly outperform a set of state-of-the-art models.
- 24 June 2022
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 17 (6) , e0269651
Open AccessTRR: Target-relation regulated network for sequential recommendation
Item co-occurrence is an important pattern in recommendation. Due to the difference in correlation, the matching degrees between the target and historical items vary. The higher the matching degree, the greater probability they co-occur. Recently, the recommendation performance has been greatly improved by leveraging item relations. As an important bond imposed by relations, these connected items should have a strong correlation in the calculation of certain measures. This kind of correlation can be the biased knowledge that benefits parameter training. Specifically, we focus on tuples containing the target item and latest relational items that have relations such as complement or substitute with the target item in user’s behavior sequence. Such close relations mean the matching degrees between relational items and historical items should be highly affected by that of the target item and historical items. For example, given a relational item having relation complement with the target item, if the target item has high matching degrees with some items in user’s behavior sequence, this complementary item should behave similarly for the co-occurrence of complementary items. Under guidance of the above thought, in this work, we propose a target-relation regulated mechanism which converts the biased knowledge of high correlation of matching degrees into a regulation. It integrates the target item and relational items in history as a whole to characterize the matching score between the target item and historical items. Experiments conducted on three real-world datasets demonstrate that our model can significantly outperform a set of state-of-the-art models.
- Open Access
- 21 January 2022
- Published by PeerJ in PeerJ Computer Science
- Vol. 8, e867
Attenuated and normalized item-item product network for sequential recommendation
Sequential recommendation has become a research trending that exploits user’s recent behaviors for recommendation. The user-item interactions contain a sequential dependency that we need to capture to better recommend. Item-item Product (IIP), which models item co-occurrence, has shown good potential by characterizing the pairwise item relationships. Generally, recent behaviors have a greater impact on the current than long-term historical behaviors. And the decaying rate of influence around infrequent behaviors is fast. However, IIP ignores such a phenomenon when considering item-item relevance and leads to suboptimal performance. In this paper, we propose an attenuated IIP mechanism which is position-aware and decays the influence of historical items at an exponential rate. Besides, In order to make up for scenarios where the influence is not in a monotonous decline trend, we add another normalized IIP mechanism to complement the attenuated IIP mechanism. It also strengthen the model’s ability in discriminating favorite items under the sparse data condition by enlarging the gap of matching degree between items. Experiments conducted on five real-world datasets demonstrate that our proposed model achieves better performance than a set of state-of-the-art sequential recommendation models.
- 21 January 2022
- Published by PeerJ in PeerJ Computer Science
- Vol. 8, e867
Open AccessAttenuated and normalized item-item product network for sequential recommendation
Sequential recommendation has become a research trending that exploits user’s recent behaviors for recommendation. The user-item interactions contain a sequential dependency that we need to capture to better recommend. Item-item Product (IIP), which models item co-occurrence, has shown good potential by characterizing the pairwise item relationships. Generally, recent behaviors have a greater impact on the current than long-term historical behaviors. And the decaying rate of influence around infrequent behaviors is fast. However, IIP ignores such a phenomenon when considering item-item relevance and leads to suboptimal performance. In this paper, we propose an attenuated IIP mechanism which is position-aware and decays the influence of historical items at an exponential rate. Besides, In order to make up for scenarios where the influence is not in a monotonous decline trend, we add another normalized IIP mechanism to complement the attenuated IIP mechanism. It also strengthen the model’s ability in discriminating favorite items under the sparse data condition by enlarging the gap of matching degree between items. Experiments conducted on five real-world datasets demonstrate that our proposed model achieves better performance than a set of state-of-the-art sequential recommendation models.
- Open Access
Improving local clustering based top-L link prediction methods via asymmetrical link clustering information
Networks can represent a wide range of complex systems, such as social, biological and technological systems. Link prediction is one of the most important problems in network analysis, and has attracted much research interest recently. Many link prediction methods have been proposed to solve this problem with various technics. We can note that clustering information plays an important role in solving the link prediction problem. In previous literatures, we find node clustering coefficient appears frequently in many link prediction methods. However, node clustering coefficient is limited to describe the role of a common-neighbor in different local networks, because it can not distinguish different clustering abilities of a node to different node pairs. In this paper, we shift our focus from nodes to links, and propose the concept of asymmetric link clustering (ALC) coefficient. Further, we improve three node clustering based link prediction methods via the concept of ALC. The experimental results demonstrate that ALC-based methods outperform node clustering based methods, especially achieving remarkable improvements on food web, hamster friendship and Internet networks. Besides, comparing with other methods, the performance of ALC-based methods are very stable in both globalized and personalized top-L link prediction tasks.
Open AccessImproving local clustering based top-L link prediction methods via asymmetrical link clustering information
Networks can represent a wide range of complex systems, such as social, biological and technological systems. Link prediction is one of the most important problems in network analysis, and has attracted much research interest recently. Many link prediction methods have been proposed to solve this problem with various technics. We can note that clustering information plays an important role in solving the link prediction problem. In previous literatures, we find node clustering coefficient appears frequently in many link prediction methods. However, node clustering coefficient is limited to describe the role of a common-neighbor in different local networks, because it can not distinguish different clustering abilities of a node to different node pairs. In this paper, we shift our focus from nodes to links, and propose the concept of asymmetric link clustering (ALC) coefficient. Further, we improve three node clustering based link prediction methods via the concept of ALC. The experimental results demonstrate that ALC-based methods outperform node clustering based methods, especially achieving remarkable improvements on food web, hamster friendship and Internet networks. Besides, comparing with other methods, the performance of ALC-based methods are very stable in both globalized and personalized top-L link prediction tasks.
- Open Access
- 1 August 2016
- Published by IOP Publishing in Journal of Statistical Mechanics: Theory and Experiment
- Vol. 2016 (8) , 083202
Predicting top-Lmissing links with node and link clustering information in large-scale networks
- 1 August 2016
- Published by IOP Publishing in Journal of Statistical Mechanics: Theory and Experiment
- Vol. 2016 (8) , 083202
Open AccessPredicting top-Lmissing links with node and link clustering information in large-scale networks
- Open Access
Link Prediction with Node Clustering Coefficient
Predicting missing links in incomplete complex networks efficiently and accurately is still a challenging problem. The recently proposed CAR (Cannistrai-Alanis-Ravai) index shows the power of local link/triangle information in improving link-prediction accuracy. With the information of level-2 links, which are links between common-neighbors, most classical similarity indices can be improved. Nevertheless, calculating the number of level-2 links makes CAR index not efficient enough. Inspired by the idea of employing local link/triangle information, we propose a new similarity index with more local structure information. In our method, local link/triangle structure information can be conveyed by clustering coefficient of common neighbors directly. The reason why clustering coefficient has good effectiveness in estimating the contribution of a common-neighbor is because that it employs links existing between neighbors of the common-neighbor and these links have the same structural position with the candidate link to this common-neighbor. Ten real-world networks drawn from five various fields are used to test the performance of our method against to classical similarity indices and recently proposed CAR index. Two estimators: precision and AUP, are used to evaluate the accuracy of link prediction algorithms. Generally speaking, our new index only performs competitively with CAR, but it is a good complement to CAR for networks with not very high LCP-corr, which is a measure to estimate the correlation between number of common-neighbors and number of links between common-neighbors. Besides, the proposed index is also more efficient than CAR index.
Open AccessLink Prediction with Node Clustering Coefficient
Predicting missing links in incomplete complex networks efficiently and accurately is still a challenging problem. The recently proposed CAR (Cannistrai-Alanis-Ravai) index shows the power of local link/triangle information in improving link-prediction accuracy. With the information of level-2 links, which are links between common-neighbors, most classical similarity indices can be improved. Nevertheless, calculating the number of level-2 links makes CAR index not efficient enough. Inspired by the idea of employing local link/triangle information, we propose a new similarity index with more local structure information. In our method, local link/triangle structure information can be conveyed by clustering coefficient of common neighbors directly. The reason why clustering coefficient has good effectiveness in estimating the contribution of a common-neighbor is because that it employs links existing between neighbors of the common-neighbor and these links have the same structural position with the candidate link to this common-neighbor. Ten real-world networks drawn from five various fields are used to test the performance of our method against to classical similarity indices and recently proposed CAR index. Two estimators: precision and AUP, are used to evaluate the accuracy of link prediction algorithms. Generally speaking, our new index only performs competitively with CAR, but it is a good complement to CAR for networks with not very high LCP-corr, which is a measure to estimate the correlation between number of common-neighbors and number of links between common-neighbors. Besides, the proposed index is also more efficient than CAR index.