Haozhe Lin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Haozhe Lin has a total of 28 co-authors. Here are the top co-authors by number of shared publications:
- Bing Bai
Tsinghua University
3 shared publications - Jia Zhang
Southern Methodist University, Dallas, TX, USA
3 shared publications - Wenxi Li
Shanghai Jiao Tong University
3 shared publications - Xiaokang Yang
Shanghai Jiao Tong University
3 shared publications - Yuchen Guo
Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China
3 shared publications - Yushun Fan
Department of Automation, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China
3 shared publications
Latest Publications
- Open Access
DyCrowd: Towards Dynamic Crowd Reconstruction from a Large-scene Video
3D reconstruction of dynamic crowds in large scenes has become increasingly important for applications such as city surveillance and crowd analysis. However, current works attempt to reconstruct 3D crowds from a static image, causing a lack of temporal consistency and inability to alleviate the typical impact caused by occlusions. In this paper, we propose DyCrowd, the first framework for spatio-temporally consistent 3D reconstruction of hundreds of individuals' poses, positions and shapes from a large-scene video. We design a coarse-to-fine group-guided motion optimization strategy for occlusion-robust crowd reconstruction in large scenes. To address temporal instability and severe occlusions, we further incorporate a VAE (Variational Autoencoder)-based human motion prior along with a segment-level group-guided optimization. The core of our strategy leverages collective crowd behavior to address long-term dynamic occlusions. By jointly optimizing the motion sequences of individuals with similar motion segments and combining this with the proposed Asynchronous Motion Consistency (AMC) loss, we enable high-quality unoccluded motion segments to guide the motion recovery of occluded ones, ensuring robust and plausible motion recovery even in the presence of temporal desynchronization and rhythmic inconsistencies. Additionally, in order to fill the gap of no existing well-annotated large-scene video dataset, we contribute a virtual benchmark dataset, VirtualCrowd, for evaluating dynamic crowd reconstruction from large-scene videos. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in the large-scene dynamic crowd reconstruction task. The code and dataset will be available for research purposes.
Open AccessDyCrowd: Towards Dynamic Crowd Reconstruction from a Large-scene Video
3D reconstruction of dynamic crowds in large scenes has become increasingly important for applications such as city surveillance and crowd analysis. However, current works attempt to reconstruct 3D crowds from a static image, causing a lack of temporal consistency and inability to alleviate the typical impact caused by occlusions. In this paper, we propose DyCrowd, the first framework for spatio-temporally consistent 3D reconstruction of hundreds of individuals' poses, positions and shapes from a large-scene video. We design a coarse-to-fine group-guided motion optimization strategy for occlusion-robust crowd reconstruction in large scenes. To address temporal instability and severe occlusions, we further incorporate a VAE (Variational Autoencoder)-based human motion prior along with a segment-level group-guided optimization. The core of our strategy leverages collective crowd behavior to address long-term dynamic occlusions. By jointly optimizing the motion sequences of individuals with similar motion segments and combining this with the proposed Asynchronous Motion Consistency (AMC) loss, we enable high-quality unoccluded motion segments to guide the motion recovery of occluded ones, ensuring robust and plausible motion recovery even in the presence of temporal desynchronization and rhythmic inconsistencies. Additionally, in order to fill the gap of no existing well-annotated large-scene video dataset, we contribute a virtual benchmark dataset, VirtualCrowd, for evaluating dynamic crowd reconstruction from large-scene videos. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in the large-scene dynamic crowd reconstruction task. The code and dataset will be available for research purposes.
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 4851-4860
SparseFormer: Detecting Objects in HRW Shots via Sparse Vision Transformer
Recent years have seen an increase in the use of gigapixel-level image and video capture systems and benchmarks with high-resolution wide (HRW) shots. However, unlike close-up shots in the MS COCO dataset, the higher resolution and wider field of view raise unique challenges, such as extreme sparsity and huge scale changes, causing existing close-up detectors inaccuracy and inefficiency. In this paper, we present a novel model-agnostic sparse vision transformer, dubbed SparseFormer, to bridge the gap of object detection between close-up and HRW shots. The proposed SparseFormer selectively uses attentive tokens to scrutinize the sparsely distributed windows that may contain objects. In this way, it can jointly explore global and local attention by fusing coarse- and fine-grained features to handle huge scale changes. SparseFormer also benefits from a novel Cross-slice non-maximum suppression (C-NMS) algorithm to precisely localize objects from noisy windows and a simple yet effective multi-scale strategy to improve accuracy. Extensive experiments on two HRW benchmarks, PANDA and DOTA-v1.0, demonstrate that the proposed SparseFormer significantly improves detection accuracy (up to 5.8%) and speed (up to 3x) over the state-of-the-art approaches.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 4851-4860
Open AccessSparseFormer: Detecting Objects in HRW Shots via Sparse Vision Transformer
Recent years have seen an increase in the use of gigapixel-level image and video capture systems and benchmarks with high-resolution wide (HRW) shots. However, unlike close-up shots in the MS COCO dataset, the higher resolution and wider field of view raise unique challenges, such as extreme sparsity and huge scale changes, causing existing close-up detectors inaccuracy and inefficiency. In this paper, we present a novel model-agnostic sparse vision transformer, dubbed SparseFormer, to bridge the gap of object detection between close-up and HRW shots. The proposed SparseFormer selectively uses attentive tokens to scrutinize the sparsely distributed windows that may contain objects. In this way, it can jointly explore global and local attention by fusing coarse- and fine-grained features to handle huge scale changes. SparseFormer also benefits from a novel Cross-slice non-maximum suppression (C-NMS) algorithm to precisely localize objects from noisy windows and a simple yet effective multi-scale strategy to improve accuracy. Extensive experiments on two HRW benchmarks, PANDA and DOTA-v1.0, demonstrate that the proposed SparseFormer significantly improves detection accuracy (up to 5.8%) and speed (up to 3x) over the state-of-the-art approaches.
- 22 August 2024
- Published by Springer Nature
- p. 392-408
SaccadeDet: A Novel Dual-Stage Architecture for Rapid and Accurate Detection in Gigapixel Images
- 22 August 2024
- Published by Springer Nature
- p. 392-408
SaccadeDet: A Novel Dual-Stage Architecture for Rapid and Accurate Detection in Gigapixel Images
- Open Access
SaccadeDet: A Novel Dual-Stage Architecture for Rapid and Accurate Detection in Gigapixel Images
The advancement of deep learning in object detection has predominantly focused on megapixel images, leaving a critical gap in the efficient processing of gigapixel images. These super high-resolution images present unique challenges due to their immense size and computational demands. To address this, we introduce 'SaccadeDet', an innovative architecture for gigapixel-level object detection, inspired by the human eye saccadic movement. The cornerstone of SaccadeDet is its ability to strategically select and process image regions, dramatically reducing computational load. This is achieved through a two-stage process: the 'saccade' stage, which identifies regions of probable interest, and the 'gaze' stage, which refines detection in these targeted areas. Our approach, evaluated on the PANDA dataset, not only achieves an 8x speed increase over the state-of-the-art methods but also demonstrates significant potential in gigapixel-level pathology analysis through its application to Whole Slide Imaging.
Open AccessSaccadeDet: A Novel Dual-Stage Architecture for Rapid and Accurate Detection in Gigapixel Images
The advancement of deep learning in object detection has predominantly focused on megapixel images, leaving a critical gap in the efficient processing of gigapixel images. These super high-resolution images present unique challenges due to their immense size and computational demands. To address this, we introduce 'SaccadeDet', an innovative architecture for gigapixel-level object detection, inspired by the human eye saccadic movement. The cornerstone of SaccadeDet is its ability to strategically select and process image regions, dramatically reducing computational load. This is achieved through a two-stage process: the 'saccade' stage, which identifies regions of probable interest, and the 'gaze' stage, which refines detection in these targeted areas. Our approach, evaluated on the PANDA dataset, not only achieves an 8x speed increase over the state-of-the-art methods but also demonstrates significant potential in gigapixel-level pathology analysis through its application to Whole Slide Imaging.
- Open Access
- 23 August 2023
- Published by Elsevier in Engineering
- Vol. 34, 70-82
The Group Interaction Field for Learning and Explaining Pedestrian Anticipation
- 23 August 2023
- Published by Elsevier in Engineering
- Vol. 34, 70-82
Open AccessThe Group Interaction Field for Learning and Explaining Pedestrian Anticipation
- 22 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Services Computing
- Vol. 16 (4) , 2786-2799
Multi-Modal Reciprocal Spatiotemporal Framework for Predicting Usage Trend of Knowledge Services
As an emerging concept, Knowledge as a Service (KaaS) aims to provide on-demand content-based (data, information, knowledge) delivery to meet the needs of users. With the prosperity of knowledge services, the prediction of the usage tendency of knowledge services has become an important and timely research topic. This study focuses on speculating the possible popularity of knowledge services in the next period of time, which can assist other downstream service tasks such as service recommendations. The interactions among knowledge services and their rich information (such as historical usage observation and text information) provide grounding for predicting the usage trend of services. However, recent spatial-temporal prediction based on graph neural networks usually depends heavily on the quality of manually created graphs, which may be expensive for knowledge services. To tackle such a limitation, this paper proposes a novel Multi-modal Reciprocal SpatioTemporal (MRST) framework, which can jointly mine spatial dependencies and model time patterns for spatiotemporal coupling prediction. Two types of Edge Inference Networks (called EIN-o and EIN-t) are designed to sufficiently discover the spatial dependencies among knowledge services based on the data of usage observation sequences and service descriptions, respectively, and generate multi-modal directed weighted knowledge service graphs. Based on these graphs, MRST integrates GCN-based spatiotemporal prediction models as backbones to make predictions. Particularly, MRST features a unique reciprocal framework. On the one hand, EINs infer and generate multi-modal graphs to serve GCNs; on the other hand, GCNs utilize such spatial dependencies to make predictions and then introduce feedback to optimize EINs. In the meantime, to facilitate reproducible research, we collect a new knowledge service dataset from Wikipedia called Wiki-EN dataset. Experiments on this real data set show that the proposed MRST framework significantly surpasses the baselines and can learn meaningful spatial dependencies outside the predefined graphic structure.
- 22 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Services Computing
- Vol. 16 (4) , 2786-2799
Multi-Modal Reciprocal Spatiotemporal Framework for Predicting Usage Trend of Knowledge Services
As an emerging concept, Knowledge as a Service (KaaS) aims to provide on-demand content-based (data, information, knowledge) delivery to meet the needs of users. With the prosperity of knowledge services, the prediction of the usage tendency of knowledge services has become an important and timely research topic. This study focuses on speculating the possible popularity of knowledge services in the next period of time, which can assist other downstream service tasks such as service recommendations. The interactions among knowledge services and their rich information (such as historical usage observation and text information) provide grounding for predicting the usage trend of services. However, recent spatial-temporal prediction based on graph neural networks usually depends heavily on the quality of manually created graphs, which may be expensive for knowledge services. To tackle such a limitation, this paper proposes a novel Multi-modal Reciprocal SpatioTemporal (MRST) framework, which can jointly mine spatial dependencies and model time patterns for spatiotemporal coupling prediction. Two types of Edge Inference Networks (called EIN-o and EIN-t) are designed to sufficiently discover the spatial dependencies among knowledge services based on the data of usage observation sequences and service descriptions, respectively, and generate multi-modal directed weighted knowledge service graphs. Based on these graphs, MRST integrates GCN-based spatiotemporal prediction models as backbones to make predictions. Particularly, MRST features a unique reciprocal framework. On the one hand, EINs infer and generate multi-modal graphs to serve GCNs; on the other hand, GCNs utilize such spatial dependencies to make predictions and then introduce feedback to optimize EINs. In the meantime, to facilitate reproducible research, we collect a new knowledge service dataset from Wikipedia called Wiki-EN dataset. Experiments on this real data set show that the proposed MRST framework significantly surpasses the baselines and can learn meaningful spatial dependencies outside the predefined graphic structure.
- 25 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Services Computing
- Vol. PP (1) , 1
Toward Knowledge as a Service (KaaS): Predicting Popularity of Knowledge Services Leveraging Graph Neural Networks
Knowledge services are becoming a rising star in the family of XaaS (Everything as a Service). In recent years, people are more willing to search for answers and share their knowledge directly over the Internet, which makes the knowledge service ecosystem prosperous. In this paper, we aim to predict the popularity of knowledge services, which will benefit the downstream industries. Toward such a task, the spatial interactions (e.g., hyperlinks in Wikipedia) and temporal observations (e.g., page views) provide crucial information. However, it is difficult to utilize this information due to: (i) complicated and different usage observations, (ii) intricate and evolutionary spatial interactions, and (iii) small world trait of the network. To tackle such issues, we propose evolutionary graph convolutional recurrent neural networks (E-GCRNNs) to simultaneously model both temporal and spatial dependencies of knowledge services from their evolving networks. Additionally, a localized mini-batch training scheme is developed, which allows the E-GCRNNs to work on large-scale knowledge services network and reduce the prediction bias caused by the small world trait. Extensive experiments on real-world datasets have demonstrated that the proposed E-GCRNNs outperform baselines in terms of prediction accuracy, especially with the prediction range being longer, while remaining computationally efficient.
- 25 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Services Computing
- Vol. PP (1) , 1
Toward Knowledge as a Service (KaaS): Predicting Popularity of Knowledge Services Leveraging Graph Neural Networks
Knowledge services are becoming a rising star in the family of XaaS (Everything as a Service). In recent years, people are more willing to search for answers and share their knowledge directly over the Internet, which makes the knowledge service ecosystem prosperous. In this paper, we aim to predict the popularity of knowledge services, which will benefit the downstream industries. Toward such a task, the spatial interactions (e.g., hyperlinks in Wikipedia) and temporal observations (e.g., page views) provide crucial information. However, it is difficult to utilize this information due to: (i) complicated and different usage observations, (ii) intricate and evolutionary spatial interactions, and (iii) small world trait of the network. To tackle such issues, we propose evolutionary graph convolutional recurrent neural networks (E-GCRNNs) to simultaneously model both temporal and spatial dependencies of knowledge services from their evolving networks. Additionally, a localized mini-batch training scheme is developed, which allows the E-GCRNNs to work on large-scale knowledge services network and reduce the prediction bias caused by the small world trait. Extensive experiments on real-world datasets have demonstrated that the proposed E-GCRNNs outperform baselines in terms of prediction accuracy, especially with the prediction range being longer, while remaining computationally efficient.
- 13 January 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Services Computing
- Vol. 15 (2) , 918-930
MSP-RNN: Multi-Step Piecewise Recurrent Neural Network for Predicting the Tendency of Services Invocation
- 13 January 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Services Computing
- Vol. 15 (2) , 918-930
MSP-RNN: Multi-Step Piecewise Recurrent Neural Network for Predicting the Tendency of Services Invocation