ZLZilong Liu

Zilong Liu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Zilong Liu has a total of 60 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Through the eyes of the machine: how foundation models represent vague cognitive regions

    by Mina Karimi, Krzysztof Janowicz, Zilong Liu, Grant McKenzie, Omid Reza Abbasi, Song Gao, Benjamin Adams, Yingjie Hu, Songlin Wang

  • Open Access

    How Geographic Information Influences Vision-Language Model Judgments of Urban Scenes

    by Yingjing Huang, Krzysztof Janowicz, Mina Karimi, Zilong Liu, Songlin Wang, Alexandra Fortacz-Lazan, Annika Suess

  • Open Access

    Golden Gate Bridge, as Always? Eliciting Prototypical Places From Autoregressive Large Language Models via Category Production

    by Zilong Liu, Krzysztof Janowicz, Mina Karimi, Meilin Shi, Ivan Majic, Alexandra Fortacz‐Lazan

    Among the various theories of categorization, Eleanor Rosch's prototype theory stands out as both influential and contested. In contrast to the classical theory of concepts, prototype theory posits that humans conceptualize the world using category structures where exemplars vary in their degree of membership. While this theory remains well‐regarded among cognitive psychologists as an account of human cognition, we know little about whether artificial agents—particularly large language models (LLMs)—also use a similar type of categorization. Given that these models have shown to mirror human‐like cognitive abilities, an important question arises: Do they also encode categories with graded membership? This question also invites a broader ethical reflection: While substantial research has documented a lack of diversity in LLM outputs, few has realized that such bias can be understood as a graded structure in the model's internal representations of categories. In this study, we explore this possibility by examining the geographical conceptualizations of 11 autoregressive LLMs. Guided by prior work that advocated empirical, inductive testing for studying geographic categories, we employ a cognitive‐psychology experiment known as category production . It prompts a model to generate exemplars of a given category. This approach yields an extensive list of category norms (i.e., typical items associated with a category) across 297 geographic categories, revealing their graded structures both encoded within individual models and collectively. Notably, we discover a strong presence of prototypical places shared across different sampling temperatures, geographic categories, and models. Furthermore, we observe and explain a lack of geographic diversity in LLM outputs through the lens of prototype theory.


  • Top Cited Papers

    The KnowWhereGraph : A Large‐Scale Geo‐Knowledge Graph for Interdisciplinary Knowledge Discovery and Geo‐Enrichment

    by Rui Zhu, Cogan Shimizu, Shirly Stephen, Colby K. Fisher, Thomas Thelen, Kitty Currier, Krzysztof Janowicz, Pascal Hitzler, Mark Schildhauer, Wenwen Li, et al.

    Global challenges such as food supply chain disruptions, public health crises, and natural hazard responses require access to and integration of diverse datasets, many of which are geospatial. Over the past few years, a growing number of (geo)portals have been developed to address this need. However, most existing (geo)portals are stacked by separated or sparsely connected data “silos” impeding effective data consolidation. A new way of sharing and reusing geospatial data is therefore urgently needed. In this work, we introduce KnowWhereGraph, a knowledge graph‐based data integration, enrichment, and synthesis framework that not only includes schemas and data related to human and environmental systems but also provides a suite of supporting tools for accessing this information. The KnowWhereGraph aims to address the challenge of data integration by building a large‐scale, cross‐domain, preintegrated, FAIR‐principles‐based, and AI‐ready data warehouse rooted in knowledge graphs. We highlight the design principles of KnowWhereGraph, emphasizing the roles of space, place, and time in bridging various data “silos.” Additionally, we demonstrate multiple use cases where the proposed geospatial knowledge graph and its associated tools empower decision‐makers to uncover insights that are often hidden within complex and poorly interoperable datasets.


  • Open Access

    The geography of evolving concepts: quantifying spatial, temporal, and semantic consequences of ontology changes

    by Meilin Shi, Krzysztof Janowicz, Zilong Liu, Mina Karimi, Ivan Majic, Alexandra Fortacz


  • Open Access

    Whose Truth? Pluralistic Geo-Alignment for (Agentic) AI

    by Krzysztof Janowicz, Zilong Liu, Gengchen Mai, Zhangyu Wang, Ivan Majic, Alexandra Fortacz, Grant Mckenzie, Song Gao


  • Open Access

    Operationalizing Geographic Diversity for the Evaluation of AI ‐Generated Content

    by Zilong Liu, Krzysztof Janowicz, Ivan Majic, Meilin Shi, Alexandra Fortacz, Mina Karimi, Gengchen Mai, Kitty Currier

    The introduction and widespread use of foundation models has accelerated the necessity of identifying geographic bias in AI‐generated content. In this respect, we operationalize geographic diversity as a countermeasure. We refine the notion of geographic diversity as the quality of including data from various places and maintaining a balance across these places in both learning and generation processes. Drawing from information theory, ecology, and prior work in AI evaluation, we provide an entropy‐based definition of geographic diversity and propose to measure geographic diversity as effective numbers of places. We apply our measurement by studying generated content from six large language models, including GPT‐3.5, GPT‐4o, Mistral 7B, Mistral Large, Claude 3 Haiku, and Claude 3.5 Sonnet. Our case study reveals that prompt variations, such as modifying concept mentions or scale mentions in a user prompt, can result in more geographic diversity in their generated content. In addition, we observe that less advanced models can generate more geographically diverse content than state‐of‐the‐art ones. Furthermore, certain places dominate the generated content of these models, yet their prominence does not reflect their real‐world counterparts. Our work stresses the importance of quantifying geographic information in AI‐generated content to support GeoAI and the broader AI evaluation in the age of foundation models.


  • Open Access

    Defining Concept Drift and Its Variants in Research Data Management: A Scientometric Case Study on Geographic Information Science

    by Meilin Shi, Krzysztof Janowicz, Zilong Liu, Mina Karimi, Ivan Majic, Alexandra Fortacz

    Just as the language used to express them, concepts constantly evolve, due to changing cultural norms, scientific progress, or major events that (re)shape our understanding of the world. For example, as humans increasingly contribute to disasters, e.g., by altering the landscape, the line between natural and anthropogenic disasters blurs. This has far‐reaching consequences for disaster response, policymaking, and also research. In research data management (RDM), evolving terms like natural disaster or global warming bring challenges for data retrieval, integration, interpretation, and reuse. This highlights the importance of studying concept drift, defined as the changes in a concept's meaning over time. While concept drift has been explored in numerous domains, including cognitive science, linguistics, semiotics, and ontology evolution, a clear definition within RDM is still lacking. This is significant as it may reduce recall during RDM‐based data retrieval. To bridge this gap, this work proposes definitions of concept drift and related phenomena, such as shifts and merges, to foster cross‐domain consensus. Using a scientometric dataset in GIScience, we apply these definitions to identify representative cases of concept drift. Our findings reveal that concepts in the GIScience domain may undergo various changes, including birth, drift, and shift, among others. These findings emphasize the need to monitor concept drift in RDM to uphold FAIR data principles, thereby improving precision and recall when searching for relevant literature or data. Lastly, we highlight the potential to measure concept drift over space (place) in addition to time alone.


  • Top Cited Papers
    Open Access

    Geography for AI sustainability and sustainability for GeoAI

    by Meilin Shi, Krzysztof Janowicz, Judith Verstegen, Kitty Currier, Nina Wiedemann, Gengchen Mai, Ivan Majic, Zilong Liu, Rui Zhu


  • Open Access

    The KnowWhereGraph ontology

    by Cogan Shimizu, Shirly Stephen, Adrita Barua, Ling Cai, Antrea Christou, Kitty Currier, Abhilekha Dalal, Colby K. Fisher, Pascal Hitzler, Krzysztof Janowicz, et al.