MASHI: Multi-Agent Vision-Language System for Home Inspection
Open Access
- 7 November 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Sensor Networks
Abstract
Home inspection evaluates the visible components of a residential building and reports their deficiencies, safety concerns, and recommended actions. Vision–language models (VLMs) can describe what a photograph shows, but inspection findings also require regulatory knowledge, system-level assessment, and familiarity with inspection practice. We present MASHI, a multi-agent vision–language system that answers home inspection questions across the ten systems of the ASHI Standard of Practice. A planner agent routes each question to an ASHI system and assigns tasks to a perception agent and a system agent. The perception agent reports visual evidence from the question’s photograph. The system agent applies the agent skill of the selected ASHI system to the question, its task, and this evidence. When needed, it retrieves text and images through multimodal retrieval-augmented generation (RAG) from a knowledge base of 45 building code editions and 18 inspection reports. For evaluation, a semi-automated pipeline with human verification turns 84 other inspection reports into 32,554 question–answer pairs. Prometheus-2 and Phi-4 rate the answers on a five-level rubric. MASHI reaches a mean judge score of 4.083, higher than those of all other compared systems, including its single-agent variant. Every difference is statistically significant.Keywords
This publication has 61 references indexed in Scilit:
- Unified framework for cross-modal alignment and thermal analysis of UAV-acquired RGB–thermal imagery for building inspectionAutomation in Construction, 2026
- EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level PlanningInternational Journal of Computer Vision, 2026
- Multimodal Alignment and Fusion: A SurveyInternational Journal of Computer Vision, 2026
- Prompt-based automation of building code information transformation for compliance checkingAutomation in Construction, 2024
- Question-answering framework for building codes using fine-tuned and distilled pre-trained transformer modelsAutomation in Construction, 2024
- Automated identification of building features with deep learning for risk analysisDiscover Applied Sciences, 2024
- Existing assets maintenance management: Optimizing maintenance procedures and costs through BIM toolsAutomation in Construction, 2023
- Integrating semantic NLP and logic reasoning into a unified system for fully-automated code checkingAutomation in Construction, 2017
- Classification of rules for automated BIM rule checking developmentAutomation in Construction, 2015
- Automatic rule-based checking of building designsAutomation in Construction, 2009