InsightX Agent: An LMM-Based Agentic Framework With Integrated Tools for Reliable X-Ray NDT Analysis
- 6 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Reliability
- Vol. 75 (00189529) , 1455-1468
Abstract
Non-destructive testing (NDT), particularly X-ray inspection, is vital for industrial quality assurance, yet existing deep-learning-based approaches often lack interactivity, interpretability, and the capacity for critical self-assessment, limiting their reliability and operator trust. To address these shortcomings, this paper proposes InsightX Agent, a novel LMM-based agentic framework designed to deliver reliable, interpretable, and interactive X-ray NDT analysis. Unlike typical sequential pipelines, InsightX Agent positions a Large Multimodal Model (LMM) as a central orchestrator, coordinating between the Sparse Deformable Multi-Scale Detector (SDMSD) and the Evidence-Grounded Reflection (EGR) tool. The SDMSD generates dense defect region proposals from multi-scale feature maps and sparsifies them through Non-Maximum Suppression (NMS), optimizing detection of small, dense targets in X-ray images while maintaining computational efficiency. The EGR tool guides the LMM agent through a chain-of-thought-inspired review process, incorporating context assessment, individual defect analysis, false positive elimination, confidence recalibration and quality assurance to validate and refine the SDMSD's initial proposals. By strategically employing and intelligently using tools, InsightX Agent moves beyond passive data processing to active reasoning, enhancing diagnostic reliability and providing interpretations that integrate diverse information sources. Experimental evaluations on the GDXray+ dataset demonstrate that InsightX Agent not only achieves a high object detection F1-score of 96.54% but also offers significantly improved interpretability and trustworthiness in its analyses, highlighting the transformative potential of LMM-based agentic frameworks for industrial inspection tasks.Keywords
All Related Versions
Funding Information
- Chengdu Science and Technology Program (2025-YF12-00016-RC)
- Research Grant Council of Hong Kong (CityU JRFS2526-1S09)
- Research Grant Council of Hong Kong (11201023)
- Research Grant Council of Hong Kong (11202224)
- National Natural Science Foundation of China (72371215)
This publication has 30 references indexed in Scilit:
- GPT-4V-AD: Exploring Grounding Potential of VQA-Oriented GPT-4V for Zero-Shot Anomaly Detection2024
- LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models2024
- High-performance deep learning segmentation for non-destructive testing of X-ray tomographyJournal of Manufacturing Processes, 2024
- Machine learning in industrial X-ray computed tomography – a reviewCIRP Journal of Manufacturing Science and Technology, 2024
- PVT v2: Improved baselines with pyramid vision transformerComputational Visual Media, 2022
- Detecting anomalies in X-ray diffraction images using convolutional neural networksExpert Systems with Applications, 2021
- Aluminum Casting Inspection using Deep Object Detection Methods and Simulated Ellipsoidal DefectsMachine Vision and Applications, 2021
- Self-attention guided model for defect detection of aluminium alloy casting on X-ray imageComputers and Electrical Engineering, 2020
- Automated detection of defects with low semantic information in X-ray images based on deep learningJournal of Intelligent Manufacturing, 2020
- GDXray: The Database of X-ray Images for Nondestructive TestingJournal of Nondestructive Evaluation, 2015