NAS-PED: Neural Architecture Search for Pedestrian Detection
- 28 November 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 47 (3) , 1800-1817
Abstract
Pedestrian detection currently suffers from two issues in crowded scenes: occlusion and dense boundary prediction, making it still challenging in complex real-world scenarios. In recent years, Convolutional Neural Networks (CNN) and Vision Transformers (ViT) have shown their superiorities in addressing these issues, where ViTs capture global feature dependency to infer occlusion parts and CNNs make accurate dense predictions by local detailed features. Nevertheless, limited by the narrow receptive field, CNNs fail to infer occlusion parts, while ViTs tend to ignore local features that are vital to distinguish different pedestrians in the crowd. Therefore, it is essential to combine the advantages of CNN and ViT for pedestrian detection. However, manually designing a specific CNN and ViT hybrid network requires enormous time and resources for trial and error. To address this issue, we propose the first Neural Architecture Search (NAS) framework specifically designed for pedestrian detection named NAS-PED, which automatically designs an appropriate CNNs and ViTs hybrid backbone for the crowded pedestrian detection task. Specifically, we formulate transformers and convolutions with various kernel sizes in the same format, which provides an unconstrained space for diverse hybrid network search. Furthermore, to search for a suitable backbone, we propose an information bottleneck based NAS objective function, which treats the process of NAS as an information extraction process, preserving relevant information and suppressing redundant information from the dense pedestrians in crowd scenes Extensive experiments on CrowdHuman, CityPersons and EuroCity Persons datasets demonstrate the effectiveness of the proposed method. Our NAS-PED obtains absolute gains of 4.0% MR $^{-2}$ and 1.9% AP over the state-of-the-art (SOTA) pedestrian detection framework on CrowdHuman datasets. For the CityPersons and EuroCity Persons datasets, the searched backbone achieves stable improvement across all three subsets, outperforming some large language-image pre-trained models. Code will be released after acceptance.Keywords
Funding Information
- National Natural Science Foundation of China (62425305)
- science and technology innovation Program of Hunan Province (2023RC1048)
- Natural Science Foundation of Hunan Province (2024JJ3013)
This publication has 75 references indexed in Scilit:
- UniNet: Unified Architecture Search with Convolution, Transformer, and MLP2022
- ViTAS: Vision Transformer Architecture Search2022
- PVT v2: Improved baselines with pyramid vision transformerComputational Visual Media, 2022
- Occlusion Handling and Multi-Scale Pedestrian Detection Based on Deep Learning: A ReviewIEEE Access, 2022
- Count- and Similarity-Aware R-CNN for Pedestrian Detection2020
- Single Path One-Shot Neural Architecture Search with Uniform Sampling2020
- Learning Efficient Single-Stage Pedestrian Detectors by Asymptotic Localization Fitting2018
- Occlusion-Aware R-CNN: Detecting Pedestrians in a Crowd2018
- Small-Scale Pedestrian Detection Based on Topological Line Localization and Temporal Feature Aggregation2018
- Ten Years of Pedestrian Detection, What Have We Learned?2015