Toward Grouping in Large Scenes With Occlusion-Aware Spatio–Temporal Transformers
- 16 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (5) , 3919-3929
Abstract
Group detection, especially for large-scale scenes, has many potential applications for public safety and smart cities. Existing methods fail to cope with frequent occlusions in large-scale scenes with multiple people, and are difficult to effectively utilize spatio-temporal information. In this paper, we propose an end-to-end framework, GroupTransformer , for group detection in large-scale scenes. To deal with the frequent occlusions caused by multiple people, we design an occlusion encoder to detect and suppress severely occluded person crops. To explore the potential spatio-temporal relationship, we propose spatio-temporal transformers to simultaneously extract trajectory information and fuse inter-person features in a hierarchical manner. Experimental results on both large-scale and small-scale scenes demonstrate that our method achieves better performance compared with state-of-the-art methods. On large-scale scenes, our method significantly boosts the performance in terms of precision and F1 score by more than 10%. On small-scale scenes, our method still improves the performance of F1 score by more than 5%. We will release the code for research purposes .Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62122058, 62171317)
- Science Fund for Distinguished Young Scholars of Tianjin Municipality (22JCJQJC00040)
This publication has 55 references indexed in Scilit:
- Self-supervised Social Relation Representation for Human Group Detection2022
- Abnormal event detection by a weakly supervised temporal attention networkCAAI Transactions on Intelligence Technology, 2021
- Recent trends in crowd analysis: A reviewMachine Learning with Applications, 2021
- Deep learning‐based action recognition with 3D skeleton: A surveyCAAI Transactions on Intelligence Technology, 2021
- Joint Learning of Social Groups, Individuals Action and Sub-group Activities in Videos2020
- Complex sequential understanding through the awareness of spatial and temporal conceptsNature Machine Intelligence, 2020
- Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition2018
- Detecting conversational groups in images and sequences: A robust game-theoretic approachComputer Vision and Image Understanding, 2016
- From Proxemics Theory to Socially-Aware Navigation: A SurveyInternational Journal of Social Robotics, 2014
- Discovering Groups of People in Images2014