Learnable Graph Matching: A Practical Paradigm for Data Association
- 6 February 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (7) , 4880-4895
Abstract
Data association is at the core of many computer vision tasks, e.g., multiple object tracking, image matching, and point cloud registration. however, current data association solutions have some defects: they mostly ignore the intra-view context information; besides, they either train deep association models in an end-to-end way and hardly utilize the advantage of optimization-based assignment methods, or only use an off-the-shelf neural network to extract features. In this paper, we propose a general learnable graph matching method to address these issues. Especially, we model the intra-view relationships as an undirected graph. Then data association turns into a general graph matching problem between graphs. Furthermore, to make optimization end-to-end differentiable, we relax the original graph matching problem into continuous quadratic programming and then incorporate training into a deep graph neural network with KKT conditions and implicit function theorem. In MOT task, our method achieves state-of-the-art performance on several MOT datasets. For image matching, our method outperforms state-of-the-art methods on a popular indoor dataset, ScanNet. For point cloud registration, we also achieve competitive results.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (U21B2042, 62072457)
- 2035 Innovation Program of CAS
This publication has 96 references indexed in Scilit:
- MOTR: End-to-End Multiple-Object Tracking with Transformer2022
- ByteTrack: Multi-object Tracking by Associating Every Detection Box2022
- Tracking Objects as Points2020
- Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking2020
- Online Multi-Object Tracking with Dual Matching Attention Networks2018
- Multi-person Tracking by Multicut and Deep Matching2016
- Performance Measures and a Data Set for Multi-target, Multi-camera Tracking2016
- SURF: Speeded Up Robust Features2006
- Probabilistic Subgraph Matching Based on Convex Relaxation2005
- The Hungarian method for the assignment problemNaval Research Logistics Quarterly, 1955