Yohanes Yohanie Fridelin Panduman
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Yohanes Yohanie Fridelin Panduman has a total of 37 co-authors. Here are the top co-authors by number of shared publications:
- Nobuo Funabiki
Graduate School of Environmental, Life, Natural Science and Technology, Okayama University, Okayama 700-8530, Japan
13 shared publications - Sritrusta Sukaridhoto
Human Centric Multimedia Research Laboratory, Department of Informatic and Computer Engineering, Politeknik Elektronika Negeri Surabaya, Surabaya 60111, Indonesia
8 shared publications - Komang Candra Brata
Department of Information and Communication Systems, Okayama University, Okayama 700-8530, Japan
6 shared publications - Evianita Dewi Fajrianti
Human Centric Multimedia Research Laboratory, Department of Informatic and Computer Engineering, Politeknik Elektronika Negeri Surabaya, Surabaya 60111, Indonesia
5 shared publications - Alfiandi Aulia Rahmadani
Department of Information and Communication Systems, Okayama University, Okayama 700-8530, Japan
3 shared publications - Noprianto Noprianto
Department of Information and Communication Systems, Okayama University, Okayama 700-8530, Japan
3 shared publications
Latest Publications
- Open Access
- 1 October 2026
- Published by Institute of Advanced Engineering and Science in IAES International Journal of Artificial Intelligence (IJ-AI)
- Vol. 15  (5) , 4819-4829
Evaluating YOLOv8-pose variants for soccer keypoint detection and homography mapping
Soccer analytics intensely relies on accurate pitch keypoint detection and homography mapping, yet soccer broadcast data poses challenges caused by camera angle variation and partial pitch coverage. This study systematically evaluates you only look once version 8 (YOLOv8)-pose variants (nano, small, medium, large, and x-large) for keypoint detection and homography mapping. These models were retrained on broadcast and scouting datasets, while homography mapping was computed using direct linear transform (DLT) and random sample consensus (RANSAC). Three metrics were used for evaluation: detection accuracy, computational efficiency, and homography reliability. Results showed that the medium variant achieved the best balance, with the lowest mean pixel error (35.67 px), competitive inference speed 49.6 frames per second (FPS), and moderate model size (52.2 MB). Furthermore, it also reached the highest homography success rate (96.2%), surpassing all other variants. These findings suggest that the medium variant is the most effective balance for real-time soccer analytics, providing robust performance in accuracy, efficiency, and reliability under diverse broadcast data. Future studies would focus on enlarging the dataset with additional keypoints, incorporating temporal smoothing, and integrating a player tracking system. For reproducibility, the dataset and code are openly available on GitHub.
- 1 October 2026
- Published by Institute of Advanced Engineering and Science in IAES International Journal of Artificial Intelligence (IJ-AI)
- Vol. 15  (5) , 4819-4829
Open AccessEvaluating YOLOv8-pose variants for soccer keypoint detection and homography mapping
Soccer analytics intensely relies on accurate pitch keypoint detection and homography mapping, yet soccer broadcast data poses challenges caused by camera angle variation and partial pitch coverage. This study systematically evaluates you only look once version 8 (YOLOv8)-pose variants (nano, small, medium, large, and x-large) for keypoint detection and homography mapping. These models were retrained on broadcast and scouting datasets, while homography mapping was computed using direct linear transform (DLT) and random sample consensus (RANSAC). Three metrics were used for evaluation: detection accuracy, computational efficiency, and homography reliability. Results showed that the medium variant achieved the best balance, with the lowest mean pixel error (35.67 px), competitive inference speed 49.6 frames per second (FPS), and moderate model size (52.2 MB). Furthermore, it also reached the highest homography success rate (96.2%), surpassing all other variants. These findings suggest that the medium variant is the most effective balance for real-time soccer analytics, providing robust performance in accuracy, efficiency, and reliability under diverse broadcast data. Future studies would focus on enlarging the dataset with additional keypoints, incorporating temporal smoothing, and integrating a player tracking system. For reproducibility, the dataset and code are openly available on GitHub.
- Open Access
- 31 May 2026
- Published by Universitas Ahmad Dahlan in International Journal of Advances in Intelligent Informatics
- Vol. 12  (2) , 472-485
Iron welding spot segmentation using Nested UNet (UNet++) enhanced with diverse convolutional modules
Welding inspection plays an essential role in manufacturing industries to ensure the integrity and quality of weld joints. However, the prevalent manual inspection procedures are inherently subjective, prone to bias, and result in inconsistent quality assessments. Therefore, there is a strong need for an automated, intelligent system capable of objectively detecting welding spots. To address this, we propose an advanced segmentation model based on deep learning and computer vision techniques, specifically utilizing a Nested UNet (UNet++) architecture enhanced by extensive architectural modifications and comprehensive hyperparameter tuning. To further optimize segmentation performance, we systematically compare various convolutional blocks integrated into the bottleneck of the network architecture. Our experimental evaluation demonstrates that employing a VGG convolutional block at the bottleneck of Nested UNet achieves the highest performance, reaching an Intersection over Union (IoU) score of 76.18% and a validation loss of 0.1713 on our collected dataset. This research advances automated welding inspection by demonstrating that an enhanced Nested UNet with an optimized VGG bottleneck significantly improves weld spot segmentation accuracy. The proposed framework provides a robust benchmark for future studies on intelligent industrial visual inspection and deep learning-based quality assurance.
- 31 May 2026
- Published by Universitas Ahmad Dahlan in International Journal of Advances in Intelligent Informatics
- Vol. 12  (2) , 472-485
Open AccessIron welding spot segmentation using Nested UNet (UNet++) enhanced with diverse convolutional modules
Welding inspection plays an essential role in manufacturing industries to ensure the integrity and quality of weld joints. However, the prevalent manual inspection procedures are inherently subjective, prone to bias, and result in inconsistent quality assessments. Therefore, there is a strong need for an automated, intelligent system capable of objectively detecting welding spots. To address this, we propose an advanced segmentation model based on deep learning and computer vision techniques, specifically utilizing a Nested UNet (UNet++) architecture enhanced by extensive architectural modifications and comprehensive hyperparameter tuning. To further optimize segmentation performance, we systematically compare various convolutional blocks integrated into the bottleneck of the network architecture. Our experimental evaluation demonstrates that employing a VGG convolutional block at the bottleneck of Nested UNet achieves the highest performance, reaching an Intersection over Union (IoU) score of 76.18% and a validation loss of 0.1713 on our collected dataset. This research advances automated welding inspection by demonstrating that an enhanced Nested UNet with an optimized VGG bottleneck significantly improves weld spot segmentation accuracy. The proposed framework provides a robust benchmark for future studies on intelligent industrial visual inspection and deep learning-based quality assurance.
- Open Access
An Extension of Input Setup Assistance Service Using Generative AI to Unlearned Sensors for the SEMAR IoT Application Server Platform
Nowadays, Internet of Things (IoT) application systems are broadly applied to various sectors of society for efficient management by monitoring environments using sensors, analyzing sampled data, and giving proper feedback. For their fast deployment, we have developed Smart Environmental Monitoring and Analysis in Real Time (SEMAR) as an integrated IoT application server platform and implemented the input setup assistance service using prompt engineering and a generative AI model to assist connecting sensors to SEMAR with step-by-step guidance. However, the current service cannot assist in connections of the sensors not learned by the AI model, such as newly released ones. To address this issue, in this paper, we propose an extension to the service for handling unlearned sensors by utilizing datasheets with four steps: (1) users input a PDF datasheet containing information about the sensor, (2) key specifications are extracted from the datasheet and structured into markdown format using a generative AI, (3) this data is saved to a vector database using chunking and embedding methods, and (4) the data is used in Retrieval-Augmented Generation (RAG) to provide additional context when guiding users through sensor setup. Our evaluation with five generative AI models shows that OpenAI’s GPT-4o achieves the highest accuracy in extracting specifications from PDF datasheets and the best answer relevancy (0.987), while Gemini 2.0 Flash delivers the most balanced results, with the highest overall RAGAs score (0.76). Other models produced competitive but mixed outcomes, averaging 0.74 across metrics. The step-by-step guidance function achieved a task success rate above 80%. In a course evaluation by 48 students, the system improved the student test scores, further confirming the effectiveness of our proposed extension.
Open AccessAn Extension of Input Setup Assistance Service Using Generative AI to Unlearned Sensors for the SEMAR IoT Application Server Platform
Nowadays, Internet of Things (IoT) application systems are broadly applied to various sectors of society for efficient management by monitoring environments using sensors, analyzing sampled data, and giving proper feedback. For their fast deployment, we have developed Smart Environmental Monitoring and Analysis in Real Time (SEMAR) as an integrated IoT application server platform and implemented the input setup assistance service using prompt engineering and a generative AI model to assist connecting sensors to SEMAR with step-by-step guidance. However, the current service cannot assist in connections of the sensors not learned by the AI model, such as newly released ones. To address this issue, in this paper, we propose an extension to the service for handling unlearned sensors by utilizing datasheets with four steps: (1) users input a PDF datasheet containing information about the sensor, (2) key specifications are extracted from the datasheet and structured into markdown format using a generative AI, (3) this data is saved to a vector database using chunking and embedding methods, and (4) the data is used in Retrieval-Augmented Generation (RAG) to provide additional context when guiding users through sensor setup. Our evaluation with five generative AI models shows that OpenAI’s GPT-4o achieves the highest accuracy in extracting specifications from PDF datasheets and the best answer relevancy (0.987), while Gemini 2.0 Flash delivers the most balanced results, with the highest overall RAGAs score (0.76). Other models produced competitive but mixed outcomes, averaging 0.74 across metrics. The step-by-step guidance function achieved a task success rate above 80%. In a course evaluation by 48 students, the system improved the student test scores, further confirming the effectiveness of our proposed extension.
- Open Access
- 5 February 2025
- Published by MDPI AG in Information
- Vol. 16  (2) , 108
Implementation of Sensor Input Setup Assistance Service Using Generative AI for SEMAR IoT Application Server Platform
For rapid deployments of various IoT application systems, we have developed Smart Environmental Monitoring and Analytical in Real-Time (SEMAR) as an integrated server platform. It is equipped with rich functions for collecting, analyzing, and visualizing various data. Unfortunately, the proper configuration of SEMAR with a variety of IoT devices can be complex and challenging for novice users, since it often requires technical expertise. The assistance of Generative AI can be helpful to solve this drawback. In this paper, we present an implementation of a sensor input setup assistance service for SEMAR using prompt engineering techniques and Generative AI. A user needs to define the requirement specifications and environments of the IoT application system for sensor inputs, and give them to the service. Then, the service provides step-by-step guidance on sensor connections, communicating board configurations, network connections, and communication protocols to the user, which can help the user easily set up the configuration to connect the relevant devices to SEMAR. For evaluations, we applied the proposal to the input sensor setup processes of three practical IoT application systems with SEMAR, namely, a smart light, water heater, and room temperature monitoring system. In addition, we applied it to the setup process of an IoT application system for a course for undergraduate students at the Insitut Bisnis dan Teknologi (INSTIKI), Indonesia. The results demonstrate the effectiveness of the proposed service for SEMAR.
- 5 February 2025
- Published by MDPI AG in Information
- Vol. 16  (2) , 108
Open AccessImplementation of Sensor Input Setup Assistance Service Using Generative AI for SEMAR IoT Application Server Platform
For rapid deployments of various IoT application systems, we have developed Smart Environmental Monitoring and Analytical in Real-Time (SEMAR) as an integrated server platform. It is equipped with rich functions for collecting, analyzing, and visualizing various data. Unfortunately, the proper configuration of SEMAR with a variety of IoT devices can be complex and challenging for novice users, since it often requires technical expertise. The assistance of Generative AI can be helpful to solve this drawback. In this paper, we present an implementation of a sensor input setup assistance service for SEMAR using prompt engineering techniques and Generative AI. A user needs to define the requirement specifications and environments of the IoT application system for sensor inputs, and give them to the service. Then, the service provides step-by-step guidance on sensor connections, communicating board configurations, network connections, and communication protocols to the user, which can help the user easily set up the configuration to connect the relevant devices to SEMAR. For evaluations, we applied the proposal to the input sensor setup processes of three practical IoT application systems with SEMAR, namely, a smart light, water heater, and room temperature monitoring system. In addition, we applied it to the setup process of an IoT application system for a course for undergraduate students at the Insitut Bisnis dan Teknologi (INSTIKI), Indonesia. The results demonstrate the effectiveness of the proposed service for SEMAR.
- Open Access
- 24 January 2025
- Published by MDPI AG in Information
- Vol. 16  (2) , 91
An Application of SEMAR IoT Application Server Platform to Drone-Based Wall Inspection System Using AI Model
Recently, artificial intelligence (AI) has been adopted in a number of Internet of Things (IoT) application systems to enhance intelligence. We have developed a ready-made server with rich built-in functions to collect, process, display, analyze, and store data from various IoT devices, the SEMAR (Smart Environmental Monitoring and Analytics in Real-Time) IoT application server platform, in which various AI techniques have been implemented to enhance its capabilities. In this paper, we present an application of SEMAR to a drone-based wall inspection system using an object detection AI model called You Only Look Once (YOLO). This system aims to detect wall cracks at high places using images taken via a camera on a flying drone. An edge computing device is installed to control the drone, sending the taken images through the Kafka system, storing them with the drone flight data, and sending the data to SEMAR. The images are analyzed via YOLO through SEMAR. For evaluations, we implemented the system using Ryze Tello for the drone and Raspberry Pi for the edge, and we evaluated the detection accuracy. The preliminary experiment results confirmed the effectiveness of the proposal.
- 24 January 2025
- Published by MDPI AG in Information
- Vol. 16  (2) , 91
Open AccessAn Application of SEMAR IoT Application Server Platform to Drone-Based Wall Inspection System Using AI Model
Recently, artificial intelligence (AI) has been adopted in a number of Internet of Things (IoT) application systems to enhance intelligence. We have developed a ready-made server with rich built-in functions to collect, process, display, analyze, and store data from various IoT devices, the SEMAR (Smart Environmental Monitoring and Analytics in Real-Time) IoT application server platform, in which various AI techniques have been implemented to enhance its capabilities. In this paper, we present an application of SEMAR to a drone-based wall inspection system using an object detection AI model called You Only Look Once (YOLO). This system aims to detect wall cracks at high places using images taken via a camera on a flying drone. An edge computing device is installed to control the drone, sending the taken images through the Kafka system, storing them with the drone flight data, and sending the data to SEMAR. The images are analyzed via YOLO through SEMAR. For evaluations, we implemented the system using Ryze Tello for the drone and Raspberry Pi for the edge, and we evaluated the detection accuracy. The preliminary experiment results confirmed the effectiveness of the proposal.
- Open Access
- 17 January 2025
- Published by MDPI AG in Electronics
- Vol. 14  (2) , 342
A Proposal of In Situ Authoring Tool with Visual-Inertial Sensor Fusion for Outdoor Location-Based Augmented Reality
In location-based augmented reality (LAR) applications, a simple and effective authoring tool is essential to create immersive AR experiences in real-world contexts. Unfortunately, most of the current tools are primarily desktop-based, requiring manual location acquisitions, the use of software development kits (SDKs), and high programming skills, which poses significant challenges for novice developers and a lack of precise LAR content alignment. In this paper, we propose an intuitive in situ authoring tool with visual-inertial sensor fusions to simplify the LAR content creation and storing process directly using a smartphone at the point of interest (POI) location. The tool localizes the user’s position using smartphone sensors and maps it with the captured smartphone movement and the surrounding environment data in real-time. Thus, the AR developer can place a virtual object on-site intuitively without complex programming. By leveraging the combined capabilities of Visual Simultaneous Localization and Mapping(VSLAM) and Google Street View (GSV), it enhances localization and mapping accuracy during AR object creation. For evaluations, we conducted extensive user testing with 15 participants, assessing the task success rate and completion time of the tool in practical pedestrian navigation scenarios. The Handheld Augmented Reality Usability Scale (HARUS) was used to evaluate overall user satisfaction. The results showed that all the participants successfully completed the tasks, taking s on average to create one AR object in a 50 m radius area, while common desktop-based methods in the literature need 1–8 min on average, depending on the user’s expertise. Usability scores reached for manipulability and for comprehensibility, demonstrating the high effectiveness in simplifying the outdoor LAR content creation process.- 17 January 2025
- Published by MDPI AG in Electronics
- Vol. 14  (2) , 342
Open AccessA Proposal of In Situ Authoring Tool with Visual-Inertial Sensor Fusion for Outdoor Location-Based Augmented Reality
In location-based augmented reality (LAR) applications, a simple and effective authoring tool is essential to create immersive AR experiences in real-world contexts. Unfortunately, most of the current tools are primarily desktop-based, requiring manual location acquisitions, the use of software development kits (SDKs), and high programming skills, which poses significant challenges for novice developers and a lack of precise LAR content alignment. In this paper, we propose an intuitive in situ authoring tool with visual-inertial sensor fusions to simplify the LAR content creation and storing process directly using a smartphone at the point of interest (POI) location. The tool localizes the user’s position using smartphone sensors and maps it with the captured smartphone movement and the surrounding environment data in real-time. Thus, the AR developer can place a virtual object on-site intuitively without complex programming. By leveraging the combined capabilities of Visual Simultaneous Localization and Mapping(VSLAM) and Google Street View (GSV), it enhances localization and mapping accuracy during AR object creation. For evaluations, we conducted extensive user testing with 15 participants, assessing the task success rate and completion time of the tool in practical pedestrian navigation scenarios. The Handheld Augmented Reality Usability Scale (HARUS) was used to evaluate overall user satisfaction. The results showed that all the participants successfully completed the tasks, taking s on average to create one AR object in a 50 m radius area, while common desktop-based methods in the literature need 1–8 min on average, depending on the user’s expertise. Usability scores reached for manipulability and for comprehensibility, demonstrating the high effectiveness in simplifying the outdoor LAR content creation process. - Open Access
- 25 December 2024
- Published by MDPI AG in Journal of Sensor and Actuator Networks
- Vol. 14  (1) , 2
Enhancing Campus Environment: Real-Time Air Quality Monitoring Through IoT and Web Technologies
Nowadays, enhancing campus environments through mitigations of air pollutions is an essential endeavor to support academic achievements, health, and safety of students and staffs in higher educational institutes. In laboratories, pollutants from welding, auto repairs, or chemical experiments can drastically degrade the air quality in the campus, endangering the respiratory and cognitive health of students and staffs. Besides, in universities in Indonesia, automobile emissions of harmful substances such as carbon monoxide (CO), nitrogen dioxide (NO2), and hydrocarbon (HC) have been a serious problem for a long time. Almost everybody is using a motorbike or a car every day in daily life, while the number of students is continuously increasing. However, people in many campuses including managements do not be aware these problems, since air quality is not monitored. In this paper, we present a real-time air quality monitoring system utilizing Internet of Things (IoT) integrated sensors capable of detecting pollutants and measuring environmental conditions to visualize them. By transmitting data to the SEMAR IoT application server platform via an ESP32 microcontroller, this system provides instant alerts through a web application and Telegram notifications when pollutant levels exceed safe thresholds. For evaluations of the proposed system, we adopted three sensors to measure the levels of CO, NO2, and HC and conducted experiments in three sites, namely, Mechatronics Laboratory, Power and Emission Laboratory, and Parking Lot, at the State Polytechnic of Malang, Indonesia. Then, the results reveal Good, Unhealthy, and Dangerous for them, respectively, among the five categories defined by the Indonesian government. The system highlighted its ability to monitor air quality fluctuations, trigger warnings of hazardous conditions, and inform the campus community. The correlation of the sensor levels can identify the relationship of each pollutant, which provides insight into the characteristics of pollutants in a particular scenario.
- 25 December 2024
- Published by MDPI AG in Journal of Sensor and Actuator Networks
- Vol. 14  (1) , 2
Open AccessEnhancing Campus Environment: Real-Time Air Quality Monitoring Through IoT and Web Technologies
Nowadays, enhancing campus environments through mitigations of air pollutions is an essential endeavor to support academic achievements, health, and safety of students and staffs in higher educational institutes. In laboratories, pollutants from welding, auto repairs, or chemical experiments can drastically degrade the air quality in the campus, endangering the respiratory and cognitive health of students and staffs. Besides, in universities in Indonesia, automobile emissions of harmful substances such as carbon monoxide (CO), nitrogen dioxide (NO2), and hydrocarbon (HC) have been a serious problem for a long time. Almost everybody is using a motorbike or a car every day in daily life, while the number of students is continuously increasing. However, people in many campuses including managements do not be aware these problems, since air quality is not monitored. In this paper, we present a real-time air quality monitoring system utilizing Internet of Things (IoT) integrated sensors capable of detecting pollutants and measuring environmental conditions to visualize them. By transmitting data to the SEMAR IoT application server platform via an ESP32 microcontroller, this system provides instant alerts through a web application and Telegram notifications when pollutant levels exceed safe thresholds. For evaluations of the proposed system, we adopted three sensors to measure the levels of CO, NO2, and HC and conducted experiments in three sites, namely, Mechatronics Laboratory, Power and Emission Laboratory, and Parking Lot, at the State Polytechnic of Malang, Indonesia. Then, the results reveal Good, Unhealthy, and Dangerous for them, respectively, among the five categories defined by the Indonesian government. The system highlighted its ability to monitor air quality fluctuations, trigger warnings of hazardous conditions, and inform the campus community. The correlation of the sensor levels can identify the relationship of each pollutant, which provides insight into the characteristics of pollutants in a particular scenario.
- Open Access
- 20 November 2024
- Published by MDPI AG in Future Internet
- Vol. 16  (11) , 429
Fully Open-Source Meeting Minutes Generation Tool
With the increasing use of online meetings, there is a growing need for efficient tools that can automatically generate meeting minutes from recorded sessions. Current solutions often rely on proprietary systems, limiting adaptability and flexibility. This paper investigates whether various open-source models and methods such as audio-to-text conversion, summarization, keyword extraction, and optical character recognition (OCR) can be integrated to create a meeting minutes generation tool for recorded video presentations. For this purpose, a series of evaluations are conducted to identify suitable models. Then, the models are integrated into a system that is modular yet accurate. The utilization of an open-source approach ensures that the tool remains accessible and adaptable to the latest innovations, thereby ensuring continuous improvement over time. Furthermore, this approach also benefits organizations and individuals by providing a cost-effective and flexible alternative. This work contributes to creating a modular and easily extensible open-source framework that integrates several advanced technologies and future new models into a cohesive system. The system was evaluated on ten videos created under controlled conditions, which may not fully represent typical online presentation recordings. It showed strong performance in audio-to-text conversion with a low word-error rate. Summarization and keyword extraction were functional but showed room for improvement in terms of precision and relevance, as gathered from the users’ feedback. These results confirm the system’s effectiveness and efficiency in generating usable meeting minutes from recorded presentation videos, with room for improvement in future works.
- 20 November 2024
- Published by MDPI AG in Future Internet
- Vol. 16  (11) , 429
Open AccessFully Open-Source Meeting Minutes Generation Tool
With the increasing use of online meetings, there is a growing need for efficient tools that can automatically generate meeting minutes from recorded sessions. Current solutions often rely on proprietary systems, limiting adaptability and flexibility. This paper investigates whether various open-source models and methods such as audio-to-text conversion, summarization, keyword extraction, and optical character recognition (OCR) can be integrated to create a meeting minutes generation tool for recorded video presentations. For this purpose, a series of evaluations are conducted to identify suitable models. Then, the models are integrated into a system that is modular yet accurate. The utilization of an open-source approach ensures that the tool remains accessible and adaptable to the latest innovations, thereby ensuring continuous improvement over time. Furthermore, this approach also benefits organizations and individuals by providing a cost-effective and flexible alternative. This work contributes to creating a modular and easily extensible open-source framework that integrates several advanced technologies and future new models into a cohesive system. The system was evaluated on ten videos created under controlled conditions, which may not fully represent typical online presentation recordings. It showed strong performance in audio-to-text conversion with a low word-error rate. Summarization and keyword extraction were functional but showed room for improvement in terms of precision and relevance, as gathered from the users’ feedback. These results confirm the system’s effectiveness and efficiency in generating usable meeting minutes from recorded presentation videos, with room for improvement in future works.
- Open Access
- 24 July 2024
- Published by MDPI AG in Electronics
- Vol. 13  (15) , 2930
Performance Investigations of VSLAM and Google Street View Integration in Outdoor Location-Based Augmented Reality under Various Lighting Conditions
The growing demand for Location-based Augmented Reality (LAR) experiences has driven the integration of Visual Simultaneous Localization And Mapping (VSLAM) with Google Street View (GSV) to enhance the accuracy. However, the impact of the ambient light intensity on the accuracy and reliability is underexplored, posing significant challenges in outdoor LAR implementations. This paper investigates the impact of light conditions on the accuracy and reliability of the VSLAM/GSV integration approach in outdoor LAR implementations. This study fills a gap in the current literature and offers valuable insights into vision-based approach implementation under different light conditions. Extensive experiments were conducted at five Point of Interest (POI) locations under various light conditions with a total of 100 datasets. Descriptive statistic methods were employed to analyze the data and assess the performance variation. Additionally, the Analysis of Variance (ANOVA) analysis was utilized to assess the impact of different light conditions on the accuracy metric and horizontal tracking time, determining whether there are significant differences in performance across varying levels of light intensity. The experimental results revealed that a significant correlation (p < ) exists between the ambient light intensity and the accuracy of the VSLAM/GSV integration approach. Through the confidence interval estimation, the minimum illuminance is needed to provide a feasible and consistent accuracy. Variations in visual references, such as wet surfaces in the rainy season, also impact the horizontal tracking time and accuracy.- 24 July 2024
- Published by MDPI AG in Electronics
- Vol. 13  (15) , 2930
Open AccessPerformance Investigations of VSLAM and Google Street View Integration in Outdoor Location-Based Augmented Reality under Various Lighting Conditions
The growing demand for Location-based Augmented Reality (LAR) experiences has driven the integration of Visual Simultaneous Localization And Mapping (VSLAM) with Google Street View (GSV) to enhance the accuracy. However, the impact of the ambient light intensity on the accuracy and reliability is underexplored, posing significant challenges in outdoor LAR implementations. This paper investigates the impact of light conditions on the accuracy and reliability of the VSLAM/GSV integration approach in outdoor LAR implementations. This study fills a gap in the current literature and offers valuable insights into vision-based approach implementation under different light conditions. Extensive experiments were conducted at five Point of Interest (POI) locations under various light conditions with a total of 100 datasets. Descriptive statistic methods were employed to analyze the data and assess the performance variation. Additionally, the Analysis of Variance (ANOVA) analysis was utilized to assess the impact of different light conditions on the accuracy metric and horizontal tracking time, determining whether there are significant differences in performance across varying levels of light intensity. The experimental results revealed that a significant correlation (p < ) exists between the ambient light intensity and the accuracy of the VSLAM/GSV integration approach. Through the confidence interval estimation, the minimum illuminance is needed to provide a feasible and consistent accuracy. Variations in visual references, such as wet surfaces in the rainy season, also impact the horizontal tracking time and accuracy. - Open Access
A User Location Reset Method through Object Recognition in Indoor Navigation System Using Unity and a Smartphone (INSUS)
To enhance user experiences of reaching destinations in large, complex buildings, we have developed a indoor navigation system using Unity and a smartphone called INSUS. It can reset the user location using a quick response (QR) code to reduce the loss of direction of the user during navigation. However, this approach needs a number of QR code sheets to be prepared in the field, causing extra loads at implementation. In this paper, we propose another reset method to reduce loads by recognizing information of naturally installed signs in the field using object detection and Optical Character Recognition (OCR) technologies. A lot of signs exist in a building, containing texts such as room numbers, room names, and floor numbers. In the proposal, the Sign Image is taken with a smartphone, the sign is detected by YOLOv8, the text inside the sign is recognized by PaddleOCR, and it is compared with each record in the Room Database using Levenshtein distance. For evaluations, we applied the proposal in two buildings in Okayama University, Japan. The results show that YOLOv8 achieved mAP@0.5 and mAP@0.5:0.95 , and PaddleOCR could extract text in the sign image accurately with an averaged CER% lower than 10%. The combination of both YOLOv8 and PaddleOCR decreases the execution time by compared to the previous method. The results confirmed the effectiveness of the proposal.Open AccessA User Location Reset Method through Object Recognition in Indoor Navigation System Using Unity and a Smartphone (INSUS)
To enhance user experiences of reaching destinations in large, complex buildings, we have developed a indoor navigation system using Unity and a smartphone called INSUS. It can reset the user location using a quick response (QR) code to reduce the loss of direction of the user during navigation. However, this approach needs a number of QR code sheets to be prepared in the field, causing extra loads at implementation. In this paper, we propose another reset method to reduce loads by recognizing information of naturally installed signs in the field using object detection and Optical Character Recognition (OCR) technologies. A lot of signs exist in a building, containing texts such as room numbers, room names, and floor numbers. In the proposal, the Sign Image is taken with a smartphone, the sign is detected by YOLOv8, the text inside the sign is recognized by PaddleOCR, and it is compared with each record in the Room Database using Levenshtein distance. For evaluations, we applied the proposal in two buildings in Okayama University, Japan. The results show that YOLOv8 achieved mAP@0.5 and mAP@0.5:0.95 , and PaddleOCR could extract text in the sign image accurately with an averaged CER% lower than 10%. The combination of both YOLOv8 and PaddleOCR decreases the execution time by compared to the previous method. The results confirmed the effectiveness of the proposal.