Zhuoheng Song
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Zhuoheng Song has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Bingchuan Jiang
Institute of Geospatial Information, Information Engineering University, Zhengzhou 450001, China
1 shared publication - Ru Yu
Institute of Geospatial Information, Information Engineering University, Zhengzhou, China
1 shared publication - Tingting Li
Institute of Geospatial Information, Information Engineering University, Zhengzhou, China
1 shared publication - Xiao Wang
Institute of Geospatial Information, Information Engineering University, Zhengzhou 450001, China
1 shared publication - Yong Sun
Information Engineering University, Zhengzhou, China;, Zhengzhou, China, China
1 shared publication
Latest Publications
- Open Access
- 18 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 19 (19391404) , 10755-10769
StreetViewGPT: A Street View Image Scene Perception and Reasoning Method Based on Multimodal Large Language Model
Multimodal large language models (MLLMs) have demonstrated promising potential in addressing real-world vision-language tasks. However, existing MLLMs exhibit relatively limited capabilities in understanding street view images, struggling to achieve multi-level semantic understanding and spatial reasoning for region-level fine-grained inference in street view images, which to some extent limits their potential in geospatial artificial intelligence (GeoAI). This study constructs an instruction fine-tuning dataset to guide MLLMs in learning spatial knowledge and scene features from street-view images, called SVGPT-Instruct. This large-scale dataset for street view understanding encompasses 430K instruction samples and 110K street view images. Ultimately, we propose the StreetViewGPT, an MLLM-based method for street view image scene perception and reasoning. The StreetViewGPT is capable of uniformly handling various street view image tasks, including time estimation (TE), weather recognition (WR), scene classification (SC), spatial distance estimation (SDE), and scene graph generation (SGG). The experimental results show that in the TE, WR, and SC tasks, the accuracy of StreetViewGPT improved by 1.33%, 21.68%, and 18.83% respectively compared to the suboptimal model. In the SDE task, the accuracy of predictions falling within the three error tolerance intervals of [75%, 125%], [50%, 150%], and [25%, 175%] reached 88.3%, 97.7%, and 99.5%, respectively, which improved by 78.2%, 78.7%, and 58.6%. In the SGG task, Recall and Mean Recall reached 16.09% and 6.84%, respectively, which improved by 12.49% and 5.4% compared to Qwen2-VL. It demonstrates that StreetViewGPT possesses multi-level semantic understanding and spatial reasoning capabilities, ranging from global semantic perception at the image level to fine-grained reasoning at the region level. The proposed method enhances the recognition and cognition capabilities for spatial scenes in street view images, offering positive implications for smart city perception and autonomous driving navigation research.
- 18 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 19 (19391404) , 10755-10769
Open AccessStreetViewGPT: A Street View Image Scene Perception and Reasoning Method Based on Multimodal Large Language Model
Multimodal large language models (MLLMs) have demonstrated promising potential in addressing real-world vision-language tasks. However, existing MLLMs exhibit relatively limited capabilities in understanding street view images, struggling to achieve multi-level semantic understanding and spatial reasoning for region-level fine-grained inference in street view images, which to some extent limits their potential in geospatial artificial intelligence (GeoAI). This study constructs an instruction fine-tuning dataset to guide MLLMs in learning spatial knowledge and scene features from street-view images, called SVGPT-Instruct. This large-scale dataset for street view understanding encompasses 430K instruction samples and 110K street view images. Ultimately, we propose the StreetViewGPT, an MLLM-based method for street view image scene perception and reasoning. The StreetViewGPT is capable of uniformly handling various street view image tasks, including time estimation (TE), weather recognition (WR), scene classification (SC), spatial distance estimation (SDE), and scene graph generation (SGG). The experimental results show that in the TE, WR, and SC tasks, the accuracy of StreetViewGPT improved by 1.33%, 21.68%, and 18.83% respectively compared to the suboptimal model. In the SDE task, the accuracy of predictions falling within the three error tolerance intervals of [75%, 125%], [50%, 150%], and [25%, 175%] reached 88.3%, 97.7%, and 99.5%, respectively, which improved by 78.2%, 78.7%, and 58.6%. In the SGG task, Recall and Mean Recall reached 16.09% and 6.84%, respectively, which improved by 12.49% and 5.4% compared to Qwen2-VL. It demonstrates that StreetViewGPT possesses multi-level semantic understanding and spatial reasoning capabilities, ranging from global semantic perception at the image level to fine-grained reasoning at the region level. The proposed method enhances the recognition and cognition capabilities for spatial scenes in street view images, offering positive implications for smart city perception and autonomous driving navigation research.