Abstract
Segment Anything Model 2 (SAM2) demonstrates outstanding performance in prompt-based visual segmentation. However, directly applying it to hyperspectral object-tracking tasks still faces numerous challenges. This article proposes TMMSAM2, a novel SAM2-based framework for hyperspectral object tracking that requires no additional training, comprising a multitemporal memory bank (MMB) and a three-dimensional constraint error mechanism (TCEM). Specifically, the MMB overcomes limitations of traditional strategies based on fixed historical frames by collaboratively integrating memory information across short-, medium-, and long-term temporal scales. Combined with feature similarity calculations and redundant frame filtering mechanisms, it constructs historical memory with optimal spatiotemporal diversity. The TCEM implements real-time quality assessment of tracking results by modeling physical constraints in three dimensions: velocity, size, and motion direction. When anomalies are detected, the system automatically triggers correction using a ViPT-based hyperspectral tracker, thereby establishing an end-to-end tracking quality assurance system. Extensive experiments on three public datasets demonstrate that our method exhibits superior performance in complex scenarios involving rapid target movement and frequent occlusions.
Funding Information
  • National Natural Science Foundation of China (42571376)
  • National Natural Science Foundation of China (62271327)
  • National Natural Science Foundation of China (42301375)
  • Department of Education of Guangdong Province (2023KCXTD029)
  • Basic and Applied Basic Research Foundation of Guangdong Province (2026A1515010204)
  • Shenzhen Science and Technology Program (RCJC20221008092731042)
  • Shenzhen Science and Technology Program (JCYJ20240813141635047)
  • AI4S Program of Shenzhen University (2026AI4S11)
  • Research Team Cultivation Program of ShenZhen University (2023JCT002)