Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking
Preprint
Abstract
This work proposes an end-to-end multi-camera 3D multi-object tracking (MOT) framework. It emphasizes spatio-temporal continuity and integrates both past and future reasoning for tracked objects. Thus, we name it "Past-and-Future reasoning for Tracking" (PF-Track). Specifically, our method adapts the "tracking by attention" framework and represents tracked instances coherently over time with object queries. To explicitly use historical cues, our "Past Reasoning" module learns to refine the tracks and enhance the object features by cross-attending to queries from previous frames and other objects. The "Future Reasoning" module digests historical information and predicts robust future trajectories. In the case of long-term occlusions, our method maintains the object positions and enables re-association by integrating motion predictions. On the nuScenes dataset, our method improves AMOTA by a large margin and remarkably reduces ID-Switches by 90% compared to prior approaches, which is an order of magnitude less. The code and models are made available at https://github.com/TRI-ML/PF-Track.Keywords
All Related Versions
This publication has 47 references indexed in Scilit:
- SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking2023
- BEVFormer: Learning Bird’s-Eye-View Representation from Multi-camera Images via Spatiotemporal Transformers2022
- PETR: Position Embedding Transformation for Multi-view 3D Object Detection2022
- MOTR: End-to-End Multiple-Object Tracking with Transformer2022
- SpOT: Spatiotemporal Modeling for 3D Object Tracking2022
- FairMOT: On the Fairness of Detection and Re-identification in Multiple Object TrackingInternational Journal of Computer Vision, 2021
- Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data2020
- End-to-End Object Detection with Transformers2020
- Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net2018
- Multiple People Tracking by Lifted Multicut and Person Re-identification2017