Multi-Modal 3D Scene Graph Updater for Shared and Dynamic Environments
Preprint
Abstract
The advent of generalist Large Language Models (LLMs) and Large Vision Models (VLMs) have streamlined the construction of semantically enriched maps that can enable robots to ground high-level reasoning and planning into their representations. One of the most widely used semantic map formats is the 3D Scene Graph, which captures both metric (low-level) and semantic (high-level) information. However, these maps often assume a static world, while real environments, like homes and offices, are dynamic. Even small changes in these spaces can significantly impact task performance. To integrate robots into dynamic environments, they must detect changes and update the scene graph in real-time. This update process is inherently multimodal, requiring input from various sources, such as human agents, the robot's own perception system, time, and its actions. This work proposes a framework that leverages these multimodal inputs to maintain the consistency of scene graphs during real-time operation, presenting promising initial results and outlining a roadmap for future research.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Nothing Stands Still: A spatiotemporal benchmark on 3D point cloud registration under large geometric and temporal changeISPRS Journal of Photogrammetry and Remote Sensing, 2025
- Clio: Real-Time Task-Driven Open-Set 3D Scene GraphsIEEE Robotics and Automation Letters, 2024
- Language-Grounded Dynamic Scene Graphs for Interactive Object Search With Mobile ManipulationIEEE Robotics and Automation Letters, 2024
- Visual Localization and Mapping in Dynamic and Changing EnvironmentsJournal of Intelligent & Robotic Systems, 2023
- PlaneSDF-Based Change Detection for Long-Term Dense MappingIEEE Robotics and Automation Letters, 2022
- Continual Learning in Real-Life ApplicationsIEEE Robotics and Automation Letters, 2022
- Online learning: A comprehensive surveyNeurocomputing, 2021
- Reinforcement learning algorithm for non-stationary environmentsApplied Intelligence, 2020
- Bayesian Landmark Learning for Mobile Robot LocalizationMachine Learning, 1998
- Modeling a dynamic environment using a Bayesian multiple hypothesis approachArtificial Intelligence, 1994