Xin Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Xin Zhang has a total of 30 co-authors. Here are the top co-authors by number of shared publications:
- Yong Li
Tsinghua University
8 shared publications - Jie Feng
Tsinghua University, China
2 shared publications - Tianhui Liu
School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing, China
2 shared publications - Tianjian Ouyang
Department of Electronic Engineering, BNRist, Tsinghua University, Beijing, China
2 shared publications - Yanxin Xi
Department of Computer Science, University of Helsinki, Helsinki, Finland
2 shared publications - Yu Shang
Department of Electronic Engineering, Tsinghua University, Beijing, China
2 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 6622-6631
CityRiSE: Reasoning Urban Socio-Economic Status in Large Vision-Language Models via Reinforcement Learning
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 6622-6631
Open AccessCityRiSE: Reasoning Urban Socio-Economic Status in Large Vision-Language Models via Reinforcement Learning
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 12519-12528
AirScape: An Aerial Generative World Model with Motion Controllability
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 12519-12528
Open AccessAirScape: An Aerial Generative World Model with Motion Controllability
- Top Cited PapersOpen Access
- 3 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 5413-5424
CityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks
- 3 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 5413-5424
Top Cited PapersOpen AccessCityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks
- Open Access
- 22 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 5342-5351
Perceiving Urban Inequality from Imagery Using Visual Language Models with Chain-of-Thought Reasoning
- 22 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 5342-5351
Open AccessPerceiving Urban Inequality from Imagery Using Visual Language Models with Chain-of-Thought Reasoning
- Open Access
- 29 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 701-704
GUI: A Comprehensive Dataset of Global Urban Infrastructure Based on Geospatial Visual Foundation Models
The substantial social and financial costs of infrastructure identification impede in-depth analyses of sustainable urban design, especially in developing countries. In this paper, we present a novel framework with interactive web visualization based on geospatial visual foundation models. Leveraging this framework, we examine the urban infrastructure information in 1,178 cities worldwide, covering 93, 088 km2 areas. Cross-validation reveals that the overall accuracy of identified infrastructure achieves 67.0%. It sheds light on the sustainable development of cities and exposes the stark inequity in urban infrastructure provision for vulnerable populations. The identified urban infrastructure dataset of this study are available at https://github.com/tsinghua-fib-lab/GUI, and the interactive web application is at https://tinyurl.com/yz7xbfy3.
- 29 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 701-704
Open AccessGUI: A Comprehensive Dataset of Global Urban Infrastructure Based on Geospatial Visual Foundation Models
The substantial social and financial costs of infrastructure identification impede in-depth analyses of sustainable urban design, especially in developing countries. In this paper, we present a novel framework with interactive web visualization based on geospatial visual foundation models. Leveraging this framework, we examine the urban infrastructure information in 1,178 cities worldwide, covering 93, 088 km2 areas. Cross-validation reveals that the overall accuracy of identified infrastructure achieves 67.0%. It sheds light on the sustainable development of cities and exposes the stark inequity in urban infrastructure provision for vulnerable populations. The identified urban infrastructure dataset of this study are available at https://github.com/tsinghua-fib-lab/GUI, and the interactive web application is at https://tinyurl.com/yz7xbfy3.
- Open Access
- 29 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 270-281
M3 LUC: Multi-modal Model for Urban Land-Use Classification
Identifying urban land-use types is crucial for effective resource management, urban planning, and sustainable development. However, classifying land use is complex due to the complexity of the city and the poor data available in undeveloped areas. In this work, we present the Multi-modal Model for Land-use Classification (M3LUC). Our model is the first to leverage the advanced Vision-Language Model (VLM) to better capture urban functionality through remote sensing data and Points of Interest (POI). We have also designed specific mechanisms to robustly and extensively tackle the modality missing and conflict to enhance transferability. Experiments conducted in four major cities in China demonstrate our model's superior performance in both transfer and non-transfer tasks, revealing its potential for broader applications.
- 29 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 270-281
Open AccessM3 LUC: Multi-modal Model for Urban Land-Use Classification
Identifying urban land-use types is crucial for effective resource management, urban planning, and sustainable development. However, classifying land use is complex due to the complexity of the city and the poor data available in undeveloped areas. In this work, we present the Multi-modal Model for Land-use Classification (M3LUC). Our model is the first to leverage the advanced Vision-Language Model (VLM) to better capture urban functionality through remote sensing data and Points of Interest (POI). We have also designed specific mechanisms to robustly and extensively tackle the modality missing and conflict to enhance transferability. Experiments conducted in four major cities in China demonstrate our model's superior performance in both transfer and non-transfer tasks, revealing its potential for broader applications.
- Open Access
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1142-1145
Health CLIP: Depression Rate Prediction Using Health Related Features in Satellite and Street View Images
Mental health is a state of mental well-being that enables people to cope with the stresses of life, realize their abilities, learn well and work well, and contribute to their community. It has intrinsic and instrumental value and is integral to our well-being, and its correlation with environmental factors has been a subject of growing interest. As the pressure of society keeps growing, depression has become a severe problem in modern cities, and finding a way to estimate depression rate is of significance to relieve the problem. In this study, we introduce a Contrastive Language-Image Pretraining (CLIP) based novel approach to predict mental health indicators, especially depression rate, through satellite and street view images. Our methodology uses state-of-the-art Multimodal Large Language Model (MLLM), GPT4-vision, to generate health related captions for satellite and street view images, then we use the generated image-text pairs to fine-tune the CLIP model, making its image encoder extract health related features such as green spaces, sports fields, and infrastructral characteristics. The fine-tuning process is employed to bridge the semantic gap between textual descriptions and visual representations, enabling a comprehensive analysis of geo-tagged images. Consequently, our methodology achieves a notable R2 value of 0.565 on prediction of depression rate in New York City with the combination of satellite and street view images. The successful deployment of Health CLIP in a real-world scenario underscores the practical applicability of our approach.
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1142-1145
Open AccessHealth CLIP: Depression Rate Prediction Using Health Related Features in Satellite and Street View Images
Mental health is a state of mental well-being that enables people to cope with the stresses of life, realize their abilities, learn well and work well, and contribute to their community. It has intrinsic and instrumental value and is integral to our well-being, and its correlation with environmental factors has been a subject of growing interest. As the pressure of society keeps growing, depression has become a severe problem in modern cities, and finding a way to estimate depression rate is of significance to relieve the problem. In this study, we introduce a Contrastive Language-Image Pretraining (CLIP) based novel approach to predict mental health indicators, especially depression rate, through satellite and street view images. Our methodology uses state-of-the-art Multimodal Large Language Model (MLLM), GPT4-vision, to generate health related captions for satellite and street view images, then we use the generated image-text pairs to fine-tune the CLIP model, making its image encoder extract health related features such as green spaces, sports fields, and infrastructral characteristics. The fine-tuning process is employed to bridge the semantic gap between textual descriptions and visual representations, enabling a comprehensive analysis of geo-tagged images. Consequently, our methodology achieves a notable R2 value of 0.565 on prediction of depression rate in New York City with the combination of satellite and street view images. The successful deployment of Health CLIP in a real-world scenario underscores the practical applicability of our approach.
- Open Access
- 30 April 2023
- Published by Association for Computing Machinery (ACM)
- p. 4150-4160
Knowledge-infused Contrastive Learning for Urban Imagery-based Socioeconomic Prediction
Monitoring sustainable development goals requires accurate and timely socioeconomic statistics, while ubiquitous and frequently-updated urban imagery in web like satellite/street view images has emerged as an important source for socioeconomic prediction. Especially, recent studies turn to self-supervised contrastive learning with manually designed similarity metrics for urban imagery representation learning and further socioeconomic prediction, which however suffers from effectiveness and robustness issues. To address such issues, in this paper, we propose a Knowledge-infused Contrastive Learning (KnowCL) model for urban imagery-based socioeconomic prediction. Specifically, we firstly introduce knowledge graph (KG) to effectively model the urban knowledge in spatiality, mobility, etc., and then build neural network based encoders to learn representations of an urban image in associated semantic and visual spaces, respectively. Finally, we design a cross-modality based contrastive learning framework with a novel image-KG contrastive loss, which maximizes the mutual information between semantic and visual representations for knowledge infusion. Extensive experiments of applying the learnt visual representations for socioeconomic prediction on three datasets demonstrate the superior performance of KnowCL with over 30% improvements on R2 compared with baselines. Especially, our proposed KnowCL model can apply to both satellite and street imagery with both effectiveness and transferability achieved, which provides insights into urban imagery-based socioeconomic prediction.
- 30 April 2023
- Published by Association for Computing Machinery (ACM)
- p. 4150-4160
Open AccessKnowledge-infused Contrastive Learning for Urban Imagery-based Socioeconomic Prediction
Monitoring sustainable development goals requires accurate and timely socioeconomic statistics, while ubiquitous and frequently-updated urban imagery in web like satellite/street view images has emerged as an important source for socioeconomic prediction. Especially, recent studies turn to self-supervised contrastive learning with manually designed similarity metrics for urban imagery representation learning and further socioeconomic prediction, which however suffers from effectiveness and robustness issues. To address such issues, in this paper, we propose a Knowledge-infused Contrastive Learning (KnowCL) model for urban imagery-based socioeconomic prediction. Specifically, we firstly introduce knowledge graph (KG) to effectively model the urban knowledge in spatiality, mobility, etc., and then build neural network based encoders to learn representations of an urban image in associated semantic and visual spaces, respectively. Finally, we design a cross-modality based contrastive learning framework with a novel image-KG contrastive loss, which maximizes the mutual information between semantic and visual representations for knowledge infusion. Extensive experiments of applying the learnt visual representations for socioeconomic prediction on three datasets demonstrate the superior performance of KnowCL with over 30% improvements on R2 compared with baselines. Especially, our proposed KnowCL model can apply to both satellite and street imagery with both effectiveness and transferability achieved, which provides insights into urban imagery-based socioeconomic prediction.