A Metadata-Grounded LLM Framework for Conversational BIM Information Retrieval in Immersive VR Environments

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Building Information Modeling (BIM) stores rich object-level metadata, yet retrieving this information typically requires navigating complex software interfaces or formal queries, and existing BIM-VR environments rarely support conversational, semantic access to this data. Large Language Models (LLMs) offer flexible natural-language interpretation but risk generating fluent, factually incorrect responses when used without grounding, which is particularly problematic for engineering decision-making. This study develops and evaluates a metadata-grounded BIM–LLM–VR framework that enables conversational BIM information retrieval inside an immersive virtual reality environment. The framework integrates a BIM-to-Unity data exchange workflow linking VR objects to BIM metadata through stable element ID, an LLM-based interpretation module that converts natural-language queries into structured metadata filters, and a deterministic retrieval module that executes these filters against verified BIM metadata rather than allowing free-form LLM generation. A case study demonstrated five successful interaction scenarios and one negative-query validation using a multi-storey BIM model, and the framework was evaluated using a 30-query benchmark validated against ground-truth BIM metadata. The system achieved a mean precision of 99.94%, mean recall of 100.00%, and mean F1-score of 99.97%, with a mean execution time of 0.347 s. These results indicate that, under a controlled metadata schema and a predefined benchmark, the framework can translate a defined set of natural-language requests into correct element-level metadata filters inside an immersive VR environment, establishing the technical soundness of the metadata-grounding architecture.
Funding Information
  • Technology Innovation Program (RS-2026-25580954)
  • Ministry of Trade, Industry and Energy
  • Korea Agency for Infrastructure Technology Advancement
  • Ministry of Land, Infrastructure and Transport under the Smart Building R&D Program (RS-2025-02532980)

This publication has 33 references indexed in Scilit: