PTP: Parallelized Tracking and Prediction With Graph Neural Networks and Diversity Sampling
Authors:
- 26 March 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (3) , 4640-4647
Abstract
Multi-object tracking (MOT) and trajectory prediction are two critical components in modern 3D perception systems that require accurate modeling of multi-agent interaction. We hypothesize that it is beneficial to unify both tasks under one framework in order to learn a shared feature representation of agent interaction. Furthermore, instead of performing tracking and prediction sequentially which can propagate errors from tracking to prediction, we propose a parallelized framework to mitigate the issue. Also, our parallel track-forecast framework incorporates two additional novel computational units. First, we use a feature interaction technique by introducing Graph Neural Networks (GNNs) to capture the way in which agents interact with one another. The GNN is able to improve discriminative feature learning for MOT association and provide socially-aware contexts for trajectory prediction. Second, we use a diversity sampling function to improve the quality and diversity of our forecasted trajectories. The learned sampling function is trained to efficiently extract a variety of outcomes from a generative trajectory distribution and helps avoid the problem of generating duplicate trajectory samples. We evaluate on KITTI and nuScenes datasets showing that our method with socially-aware feature learning and diversity sampling achieves new state-of-the-art performance on 3D MOT and trajectory prediction. Project website is: http://www.xinshuoweng.com/projects/PTP.Keywords
All Related Versions
Funding Information
- Qualcomm
This publication has 33 references indexed in Scilit:
- Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks2018
- Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net2018
- End-to-end Learning of Multi-sensor 3D Tracking by Detection2018
- Learning Detection with Diverse Proposals2017
- DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents2017
- Social LSTM: Human Trajectory Prediction in Crowded Spaces2016
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012
- A new model for learning in graph domains2006
- An algorithm directly finding the K most probable configurations in Bayesian networksInternational Journal of Approximate Reasoning, 1994
- The Hungarian method for the assignment problemNaval Research Logistics Quarterly, 1955