LongShortNet: Exploring Temporal and Semantic Features Fusion in Streaming Perception
Authors:
- Jun-Yan HeJHJun-Yan He
Preprint
Abstract
Streaming perception is a task of reporting the current state of autonomous driving, which coherently considers the latency and accuracy of autopilot systems. However, the existing streaming perception only uses the current and adjacent two frames as input for learning the movement patterns, which cannot model actual complex scenes, resulting in failed detection results. To solve this problem, we propose an end-to-end dual-path network dubbed LongShortNet, which captures long-term temporal motion and calibrates it with short-term spatial semantics for real-time perception. Moreover, we investigate a Long-Short Fusion Module (LSFM) to explore spatiotemporal feature fusion, which is the first work to extend long-term temporal in streaming perception. We evaluate the proposed LongShortNet and compare it with existing methods on the benchmark dataset Argoverse-HD. The results demonstrate that the proposed LongShortNet outperforms the other state-of-the-art methods with almost no extra computational cost.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- PTSEFormer: Progressive Temporal-Spatial Enhanced TransFormer Towards Video Object Detection2022
- Efficient One-Stage Video Object Detection by Exploiting Temporal Consistency2022
- Mining Inter-Video Proposal Relations for Video Object Detection2020
- Towards Streaming Perception2020
- Video Object Detection with an Aligned Spatial-Temporal Memory2018
- Object Detection in Video with Spatiotemporal Sampling Networks2018
- Fully Motion-Aware Network for Video Object Detection2018
- Towards High Performance Video Object Detection2018
- Deep Feature Flow for Video Recognition2017
- Microsoft COCO: Common Objects in Context2014