ETR: Enhancing Taillight Recognition via Transformer-Based Video Classification
- 11 December 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 26 (2) , 2721-2733
Abstract
In autonomous driving, efficiently and accurately recognizing taillight states using dashcams is essential for interpreting other vehicles’ intentions. Recent video-based taillight recognition methods outperform earlier image-based approaches. However, they face challenges in efficiently integrating spatiotemporal information and managing high computational costs. In this paper, we introduce ETR, an accurate and efficient Transformer-based video classification model designed to enhance taillight recognition. Specifically, we first design a lightweight backbone to extract temporal and spatial features from videos and generate queries with prior information. Next, we develop a hierarchical progressive Transformer decoder that integrates feature maps from different levels of the backbone to enhance the model’s global information. Finally, we employ a classification head to predict the taillight state of the video. Additionally, we introduce a public dataset, ETR-Taillights, to address the current lack of open datasets for vehicle taillight recognition. The dataset contains 28,799 dashcam video clips, making it the largest public taillight recognition dataset. Experiments show that our method achieves a 91.69% F-measure on the ETR-Taillights dataset, surpassing the latest taillight recognition methods, VIF by 6.94% and CNN-LSTM by 10.82%. Additionally, it achieves an inference speed of 45.06 FPS, being 3.6 times faster than VIF. Furthermore, we conduct real-world road tests to demonstrate our method’s robustness and effectiveness in practical scenarios. Our model and dataset are available at https://github.com/yang590/ vehicle-taillight.Keywords
Funding Information
- Scientific Research Project of Anhui Higher Education Institutions of China (2023AH050911)
This publication has 36 references indexed in Scilit:
- SSIR: Spatial shuffle multi-head self-attention for Single Image Super-ResolutionPattern Recognition, 2023
- Video Swin Transformer2022
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows2021
- Multiscale Vision Transformers2021
- Video Transformer Network2021
- A novel keyframe extraction method for video classification using deep neural networksNeural Computing and Applications, 2021
- SlowFast Networks for Video Recognition2019
- DeepSignals: Predicting Intent of Drivers Through Visual Signals2019
- Compressed Video Action Recognition2018
- A Robust Algorithm for the Detection of Vehicle Turn Signals and Brake Lights2012