Towards General Urban Monitoring with Vision-Language Models: A Review, Evaluation, and a Research Agenda
Authors:
- André TorneiroATAndré Torneiro
- Diogo MonteiroDMDiogo Monteiro
- Paulo NovaisPaulo Novais
- Pedro Rangel HenriquesPHPedro Rangel Henriques
- Nuno F. RodriguesNuno F. Rodrigues
Preprint
Abstract
Urban monitoring of public infrastructure (such as waste bins, road signs, vegetation, sidewalks, and construction sites) poses significant challenges due to the diversity of objects, environments, and contextual conditions involved. Current state-of-the-art approaches typically rely on a combination of IoT sensors and manual inspections, which are costly, difficult to scale, and often misaligned with citizens' perception formed through direct visual observation. This raises a critical question: Can machines now "see" like citizens and infer informed opinions about the condition of urban infrastructure? Vision-Language Models (VLMs), which integrate visual understanding with natural language reasoning, have recently demonstrated impressive capabilities in processing complex visual information, turning them into a promising technology to address this challenge. This systematic review investigates the role of VLMs in urban monitoring, with particular emphasis on zero-shot applications. Following the PRISMA methodology, we analyzed 32 peer-reviewed studies published between 2021 and 2025 to address four core research questions: (1) What urban monitoring tasks have been effectively addressed using VLMs? (2) Which VLM architectures and frameworks are most commonly used and demonstrate superior performance? (3) What datasets and resources support this emerging field? (4) How are VLM-based applications evaluated, and what performance levels have been reported?Keywords
All Related Versions
This publication has 40 references indexed in Scilit:
- Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection2024
- Building usage prediction in complex urban scenes by fusing text and facade features from street view images using deep learningBuilding and Environment, 2024
- Zero-shot urban function inference with street view images through prompting a pretrained vision-language modelInternational Journal of Geographical Information Science, 2024
- Mixed land use measurement and mapping with street view images and spatial context-aware prompts via zero-shot multimodal learningInternational Journal of Applied Earth Observation and Geoinformation, 2023
- Few-Shot Object Detection for Classifying Reported Urban Problems2023
- Counterfactual Balancing Feature Alignment for Few-Shot Cross-Domain Scene ParsingIEEE Access, 2023
- Deep-learning generation of POI data with scene imagesISPRS Journal of Photogrammetry and Remote Sensing, 2022
- The Mapillary Traffic Sign Dataset for Detection and Classification on a Global Scale2020
- Visual Urban Perception with Deep Semantic-Aware Network2018
- The Urbanization Process and Economic Growth: The So-What QuestionJournal of Economic Growth, 2003